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Dear Attorney-General and Assistant Minister Stoker, 
Human Rights and Technology Final Report 2021 


| am pleased to present to you the Australian Human Rights Commission's Human Rights and 
Technology Final Report. 


This Report is the culmination of a major project on new and emerging technologies, including 
artificial intelligence. The Report reflects the Commission's extensive public consultation 
regarding the impact of new technologies on human rights. 


Australians have always been resourceful, and we are living in a time of unprecedented 
technological transformation. Australia can seize the benefits that new technology offers, 

while effectively guarding against the risks of harm, especially to our human rights. This Report 
presents a roadmap to achieve this aim by modernising a number of laws, government policies, 
education and resourcing. 


This project and Report were conducted under section 11 of the Australian Human Rights 
Commission Act 1986 (Cth). | look forward to discussing the Report and its recommendations 
with you. 


Yours sincerely, 


Lisaa Sadis 


Edward Santow 
Human Rights Commissioner 


Australian Human Rights Commission 
T: 02 9284 9600 
Website: www.humanrights.gov.au 
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Human Rights Commissioner 
Australian Human Rights Commission 


1. Commissioner’s Foreword 


We are living through a time of unprecedented technological growth. It is reshaping 
our world. 


A modern smartphone is millions of times more powerful than the most 
sophisticated computer that guided the Apollo 11 spacecraft to the moon. It is more 
powerful even than the best supercomputers of the 1990s, such as IBM's famous 
Deep Blue. 


New technology can improve our lives. Artificial intelligence (Al) is enabling 

big strides in critical areas such as healthcare and service delivery. Digital 
communications technology can provide new ways for people with disability and 
others to connect and participate in community life—something that is more 
important than ever during the COVID-19 pandemic. 


At the same time, there are real and profound threats. Poor technology design can 
exclude people with disability from work, services and the economy. 


The use of Al can also open the door to a range of dystopian threats—from mass 
surveillance to unaccountable exercise of power by government and corporations. 
We have seen worrying uses of Al in areas as diverse as social security, policing and 
home loans. At its worst, the use of Al can result in unfairness or even discrimination 
based on characteristics such as race, age or gender. 


We need to ask a crucial question: can we harness technology's positive potential to 
deliver the future we want and need, or will it supercharge society’s worst problems? 


The decisions we make now will provide the answer. 


Australians want technology that is safe, fair and reliable. With the right settings— 
in law, policy, education and funding—the Australian Government and the 

private sector can build a firm foundation of public trust in new technology that is 
developed and used here. 


Australia should pursue innovation that reflects our liberal democratic values. We 
should approach new and emerging technologies by being consultative, inclusive 
and accountable, and by embedding robust human rights safeguards. This Final 
Report sets out a roadmap for achieving this goal. 


As Prime Minister Scott Morrison has rightly said, ‘the rules that exist in the real 
world need to exist in the digital world.” The law's protective umbrella should apply 
to the use of Al just as it does to everything else. 
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Commissioner's foreword 


To achieve this, we must modernise our regulatory system to ensure that Al-informed decision making 
is lawful, transparent, explainable, responsible, and subject to appropriate human oversight, review and 
intervention. 


Some targeted reform is needed, especially in high-risk areas. Stronger laws are needed, for example, to 
protect the community from misuse of facial recognition and other biometric technology. 


But the law cannot be the only answer. This Report contains a series of recommendations that would help 
industry, researchers, civil society and government achieve the shared goal of human-centred Al—through 
policy, co-regulation, education and training. 


This Report recommends the creation of a new Al Safety Commissioner to help lead Australia’s transition 
to an Al-powered world. This new body would be a key source of expertise on Al, building capacity among 
regulators, providing guidance to government and the private sector, and providing independent advice to 
policy makers and Parliament. 


As technology becomes essential for all aspects of life—in everything from government services to 
employment—accessibility is vital. This Report includes recommendations to ensure that noone is left 
behind as Australia continues its digital transformation. 


The Australian Human Rights Commission thanks everyone who participated in our two-stage consultation 
process: community members, civil society representatives, industry, academia and government. This input 
has been crucial in identifying problems and solutions. 


We also pay tribute to the Commission’s major partners in this Project: Australia’s Department of Foreign 
Affairs and Trade; Herbert Smith Freehills; LexisNexis; and the University of Technology Sydney. In addition, 
we thank the Digital Transformation Agency and the World Economic Forum for their significant support. 
The Commission also acknowledges the generosity of its Expert Reference Group, who have given invaluable 
strategic guidance and technical expertise. 


The COVID-19 pandemic has accelerated our reliance on new technology—for everything from health care 
to remaining connected in a time of social distancing. This experience should reinforce the importance of 
our task: to ensure new technology is safe, reliable and enhances the lives of everyone. 


Pisa Seeds 


Edward Santow 
Human Rights Commissioner 


March 2021 


2. Executive summary 


Now is a critical time. Australia is on a pathway towards becoming a leading digital 
economy. There is widespread recognition that we should walk this path while upholding 
human rights and responding to ethical risks.? This Final Report aims to help Australia 
meet this challenge. 


Australia should pursue innovation that holds true to our liberal democratic values. 
This means we should approach new and emerging technologies by being consultative, 
inclusive and accountable, with robust human rights safeguards. 


This Report provides a roadmap for how Australia should protect and promote human 
rights in a time of unprecedented change in how we develop and use new technologies. 


The Australian Human Rights Commission is Australia’s national human rights institution. 
The Commission is independent and impartial. It aims to promote and protect human 
rights in Australia. 


The recommendations in this Report are informed by the Commission's expertise, our 
research and extensive public consultation with the community, government, industry 
and academia. All views are the Commission's, and the Commission is responsible for 
this Report and other Project outputs and statements. 


2.l Context 


This era—sometimes referred to as the Fourth Industrial Revolution—brings 
extraordinary opportunities and risks, including for our human rights. The challenge for 
government is to show leadership, by creating an environment to enable Australia to 
seize the opportunities while addressing the risks. 


While there are many pressing human rights issues raised by the rise of new 
technologies,? this Project has focused on two areas in particular. 


The first is the use of artificial intelligence (Al) in an almost limitless range of decision 
making—in areas as diverse as government services, recruitment and the criminal 
justice system. This Report shows how Al can improve decision making through better 
use of data, and also how it can cause harm. The Commission sets out a suite of 
recommendations that respond to this two-sided phenomenon. 


The second area of focus is how people with disability experience digital communication 
technologies—such as information and communication technology, the Internet of 
Things (loT), virtual reality and augmented reality. Digitisation affects almost every area 
of life, and so this Report sets out ways to improve accessibility of the goods, services 
and facilities that use these technologies for the whole community, including people with 
disability. 


The final phase of this Project was conducted during the COVID-19 pandemic. Al can 
help address this public health emergency, through its use in direct medical research, 
and also in measures such as contact tracing to help model disease transmission and 
control. On the other hand, some existing inequalities have intensified and become 
more urgent. For instance, as we practise social distancing—which increases our 
reliance on technology for work, accessing services and to stay connected—inaccessible 
technologies can be even more problematic for people with disability. 
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2. Executive summary 


International law requires human rights to be 
upheld in any emergency or time of crisis, with 
special provisions that enable an appropriate 
response to public health, public order and other 
such threats. 


2.2 A human rights approach to 
new technologies 


Human rights are set out in international and 
domestic laws. International human rights law 
requires nation states to respect, protect and fulfil 
human rights, and to uphold the principle that ‘all 
human beings are born free and equal in dignity 
and rights’. 


Human rights are universal, meaning that they 
apply to everyone. They are indivisible, meaning 
that all human rights have equal status. They 

are interdependent and interrelated, meaning the 
improvement of one human right can facilitate the 
advancement of others. Likewise, the deprivation 
of one right can also negatively affect other human 
rights. 


While there are sometimes complex inter- 
relationships between different rights, governments 
must ensure everyone’s human rights are 
protected.° 


Australia is a signatory to seven core human rights 
treaties, which include civil and political rights, and 
economic, social and cultural rights.° Australia has 
voluntarily agreed to comply with human rights 
standards and to integrate them into domestic law, 
policy and practice. 


A human rights approach builds human rights into 
all aspects of law, policy development and decision 
making. Applying this approach to the development 
and use of new technologies is increasingly 
common internationally,’ with a growing number 
of experts emphasising the importance of human 
rights law in analysing the social impact of 
technology.® 


Some international initiatives use human rights 

as the primary lens,’ or one of several lenses,'° 
through which to view the development and use of 
new technologies. Given that the technologies are 
new, the application of human rights and other laws 
in this area is also developing." 


Some of the Australian and international 
organisations that support a human rights 
approach to the development and use of new 
technologies include: 


United Nations (UN) mechanisms, such 
as the UN High Commissioner for Human 
Rights and the Special Rapporteurs on 
extreme poverty and human rights,’ 
privacy and freedom of expression’? and 
racial discrimination” 


UN Guiding Principles on Business and 
Human Rights'® 


multilateral international bodies, such as 
the Organisation for Economic Cooperation 
and Development (OECD)'® and G20” 


the Australian Government including 
Australia’s A/ Ethics Principles and the 
Discussion Paper for an Al Action Plan for 
All Australians'® 


comparable liberal democracies, such as 
Canada and France’? 


some leading multinational corporations” 


non-government and industry organisations 
working at the intersection of human rights 
and new technologies.” 


The Commission uses the ‘PANEL framework’ 
to apply this human rights approach to the 
development and use of new technologies.” 


PANEL Principles: 


A human rights 
approach to new 
technologies 


Participation 


Ensure participation in decision 

making of stakeholders affected by 

new technologies, including the public, 
affected groups and civil society, experts, 
and decision makers. 


Accountability 


This requires effective monitoring of 
compliance with human rights standards 
by government and non-state actors, and 
mechanisms to enforce rights. 


Non-discriminatory and equality 


Anti-discrimination law principles should 
be applied to the development and use of 
new technologies, considering especially 
the needs of people who are vulnerable. 


Empowerment and capacity building 


The community needs to understand the 
impact of new technologies on their lives 
and have knowledge of, and access to, a 
review process and/or remedy. 


Legality 


The law should recognise that human 
rights are legally enforceable, including in 
the use of new technology. 


In addition to an effective legal regime, with clear 
and well-enforced legal obligations, there is growing 
recognition that more attention is needed at the 
stage of design and development to ensure that 
human rights are ‘baked into’ products, services and 
facilities that use new and emerging technologies. 


To this end, the Commission supports ‘human 
rights by design’, which is a systematic process 
for embedding human rights standards and 
compliance into the design, development and 
deployment of new technologies. This involves 
considering, at the point of research and 
development, the human rights implications of 
a possible use of new technology, with a view to 
designing in rights protections.” 


The objective is to produce tech-powered products, 
services and facilities that are accessible to people 
with disability, as well as protecting other human 
rights. Ahuman rights by design approach is similar 
to other design-led approaches, such as privacy and 
safety by design.” 


2.3 Overview of this Report 


The Report is divided into four parts. 


Part A: A national strategy on new and 
emerging technologies 


Part A considers Australia’s overarching approach 
to new technologies, and especially the role of the 
Australian Government's Digital Australia Strategy.” 
The Commission commends the Government on its 
ongoing process to develop this Strategy. 


The Commission recommends that the Digital 
Australia Strategy set out a vision for responsible 
innovation. The Strategy should advance human 
rights in the development and use of new and 
emerging technologies. It should contain practical 
measures focused on: regulation; education, 
training and capacity building; and funding and 
investment. 
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2. Executive summary 


Part B: Artificial intelligence 


Part B, comprising Chapters 4-9, considers how 

Al is changing the way important decisions are 
made by government and the private sector—with 
significant implications for how human rights are 
fulfilled. 


As Al disrupts our economic, social and 
governmental systems, Australian governments are 
seeking to harness the potential of Al to improve 
services, products and more. At the same time, 
there is growing public concern that Al can be used 
in ways that cause harm. 


While some uses of Al do not meaningfully engage 
people's rights, this Project has focused on ‘Al- 
informed decision making’—that is, the use of 

Al to make decisions that have legal or similarly 
significant effects on individuals. Such decisions 
are likely also to engage human rights in a more 
profound way. 


The Commission makes recommendations to 
ensure human rights are protected where Al is 
used in decision making. In particular, the Report 
has concluded that Al-informed decision making 
should be lawful, transparent, explainable, used 
responsibly, and subject to appropriate human 
oversight, review and intervention. 


Chapter 4 explores the critical link between 
effective accountability and upholding human 
rights in the use of Al-informed decision making. 
The chapter sets out three key principles for 
accountable use of Al: 


e The Australian Government should comply 
with human rights in its own use of Al, and 
it should ensure effective human rights 
safeguards apply to all entities that use Al. 


Al-informed decision-making systems 
should be tested before they are used in 
ways that could harm individuals, and those 
systems should continue to be monitored 
as they operate. 


An individual affected by an Al-informed 
decision should be given reasons for 

the decision, and they should be able to 
challenge the decision if they consider it to 
be wrong or unlawful. 


These principles are applied in the remaining 
chapters in Part B. 


Chapter 5 focuses on reform to ensure that 
government complies with human rights, and acts 
accountably, when it uses Al to make administrative 
decisions. The chapter recommends: 


e that the Australian Government be required 
to undertake a human rights impact 
assessment before it uses Al to make 
administrative decisions 


greater transparency in the use of Al by 
government. In particular, individuals 
should be notified where government uses 
Al in administrative decision making, and 
there should be clearer rules regarding 
when and how government bodies 

will provide reasons for Al-informed 
administrative decisions 


e that independent merits review be available 
for all Al-informed administrative decisions. 


Chapter 6 focuses on accountability for Al- 
informed decisions made by non-government entities 
such as corporations. 


While the law on administrative decision making 
by government is generally more prescriptive, 
accountability remains vitally important when 
corporations and other non-government entities 
use Al to make decisions. The Commission 
recommends that: 


e individuals are notified of the use of Al in 
this type of decision making 


non-government entities be encouraged 
and supported to provide reasons or an 
explanation for their Al-informed decisions 


legislation prohibit the use of ‘black box’ 
or opaque Al in decision making by non- 
government entities 


individuals be given a practical means of 
appealing to a body that can review Al- 
informed decisions by non-government 
entities. 


Chapter 7 addresses the role of co- and self- 
regulation to incentivise better human rights 
protections in the development and use of Al- 
informed decision-making systems. The aim is to 
complement and support legal regulation, and to 
create better Al-informed decision-making systems 
that avoid human rights and other problems 
occurring in the first place. 


The measures recommended in this chapter 
include: 


standards and certification for the use of Al 
in decision making 


‘regulatory sandboxes’ that allow for 
experimentation and innovation in how Al 
is used and regulated 


human rights impact assessments, 
including public consultation at the 
development stage of Al-informed decision 
making 


rules for government procurement of 
Al-informed decision-making tools and 
systems 


enhancing the role of human oversight 
and intervention in the monitoring and 
evaluation of Al-informed decision making. 


Chapter 8 explores the problem of ‘algorithmic 
bias’, which arises where an Al-informed decision- 
making tool produces outputs that result in 
unfairness. Often this is caused by some forms of 
statistical bias. Algorithmic bias has arisen in Al- 
informed decision making in the criminal justice 
system, advertising, recruitment, healthcare, 
policing and elsewhere. 


Algorithmic bias can sometimes have the effect 
of obscuring and entrenching unfairness or 
even unlawful discrimination in decision making. 
This chapter recommends greater guidance for 
government and non-government bodies in 
complying with anti-discrimination law in the 
context of Al-informed decision making. 


Finally, Chapter 9 focuses on the use of facial 
recognition and other biometric technologies. 


There is strong and growing community concern 
regarding some forms of facial recognition 
technology, which can be prone to high error rates, 
especially for certain racial and other groups. 
Where these biometric technologies are used in 
high-stakes decision making, such as policing, 
errors in identification can increase the risk of 
human rights infringement and broader injustice. 
Moreover, as the use of biometric technologies 
increases, there is concern about the impact on 
individual privacy, including through harmful 
surveillance. 


This chapter recommends law reform to provide 
stronger, clearer and more targeted human rights 
protections regarding the development and 

use of biometric technologies, including facial 
recognition. Until these protections are in place, 
the Commission recommends a moratorium on 
the use of biometric technologies, including facial 
recognition, in high-risk areas of decision making. 


Part C: Supporting effective regulation 


Part C recommends the establishment of an Al 
Safety Commissioner as an independent statutory 
office to provide technical expertise and capacity 
building on the development and use of Al. 


An Al Safety Commissioner could build public trust 
in the safe use of Al by: 


* providing expert guidance to government 
agencies and the private sector on how to 
comply with laws and ethical standards 
regarding the development and use of Al 


working collaboratively to build the capacity 
of regulators and the broader ‘regulatory 
ecosystem’ to adapt and respond to the 
rise of Al in their respective areas of 
responsibility 


monitoring trends in the use of Al in 
Australia and overseas, providing robust, 
independent and expert advice to 
legislators and policy makers with a view to 
addressing risks and taking opportunities 
connected to the rise of Al. 
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2. Executive summary 


Part D: Accessible technology 


Part D, comprising Chapters 11-15, makes 
recommendations to improve the accessibility 
of goods, services and facilities that use Digital 
Communication Technologies for people with 
disability. 


Good technology design can enable the 
participation of people with disability as never 
before—from the use of real-time live captioning 

to reliance on smart home assistants. On the other 
hand, poor design can cause significant harm, 
reducing the capacity of people with disability 

to participate in activities that are central to the 
enjoyment of their human rights, and their ability to 
live independently. 


Chapter 11 explains how the accessibility of new 
technology, and especially of Digital Communication 
Technology, is an enabling right for people with 
disability—critical to the enjoyment of a range of 
other civil, political, economic, social and cultural 
rights. The remaining chapters in Part D apply this 
principle to a number of specific contexts. 


Chapter 12 considers the functional accessibility 
of goods, services and facilities that rely on Digital 
Communication Technology—that is, the ability to 
use these things in practice. 


Problems in this area commonly relate to the 
user interface of tech-enabled products being 
designed in a way that excludes people with one 
or more disabilities. Functional accessibility needs 
to be incorporated in the design of hardware and 
software, and in any updates or new versions, 

of goods, services and facilities that use Digital 
Communication Technologies. 


This chapter recommends three key changes to 
improve functional accessibility: 


the creation of a new Disability Standard, 
focused on Digital Communication 
Technology, under the Disability 
Discrimination Act 1992 (Cth) 


amending government procurement rules 
to require accessible goods, services and 
facilities 


improving private sector use of accessible 
Digital Communication Technology. 


Chapter 13 deals with the accessibility of audio- 
visual news, information and entertainment 
content. 


The 21% century has seen a massive expansion 

in the ways such content is delivered. Traditional 
media, such as broadcast television and radio, have 
been supplemented by an ever-increasing range of 
media, including subscription television, video and 
online platforms. 


Accessibility features—especially captioning and 
audio description—can make this audio-visual 
content accessible for people with disability, 
especially people with disabilities affecting their 
vision or hearing. The Commission considers 
reform is needed to ensure that traditional 

and newer media respect the right of people 
with disability to receive news, information and 
entertainment content in ways that they can 
understand. 


This chapter recommends reforms to law and policy 
to facilitate: 


e increased audio description and captioning 
for broadcasting services, as well as video, 
film and online platforms 


reliable accessible information during 
emergency and important public 
announcements 


better monitoring of compliance with 
accessibility requirements and voluntary 
targets for the distribution of audio-visual 
content. 


Chapter 14 addresses the experience of people 
with disability in obtaining goods, services 

and facilities that use Digital Communication 
Technologies. 


Availability can be reduced where people with 
disability cannot afford, or do not know about, such 
goods, services and facilities. This problem can 

be made worse by socio-economic disadvantage, 
which is more common among people with 
disability. Exclusion can worsen inequality and 
disadvantage for people with disability. 


This chapter recommends three key measures in 
this area: 


The Commission recommends applying a 
‘human rights by design’ approach to Digital 


E CAP ; Communication Technologies through: 
e the provision of accessible information on 


how goods, services and facilities can be 
used by people with disability 


more accessible broadband internet by 
introducing a concessional rate for people 
with disability 


National Disability Insurance Scheme 
(NDIS) funding to improve access to Digital 
Communication Technologies for people 
with disability. 


Chapter 15 addresses the role of design, education 
and capacity building in promoting accessible 
Digital Communication Technologies. 
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the Australian Government adopting, 
promoting and modelling good practice 


incorporating this approach into education 
and training—especially in the areas of 
science, technology, engineering and 
mathematics 


upskilling the existing workforce via 
ongoing professional development for 
those involved in designing and developing 
with Digital Communication Technologies 


practical capacity building and accreditation 
measures for accessible and inclusive 
practices. 


16 
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regulation 
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use of Al regulators 
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2.4 Key terminology 


(a) Artificial intelligence 


This Report uses the term ‘artificial intelligence’ (Al) 
to refer to a cluster of technologies and techniques, 
which include some forms of automation, machine 
learning, algorithmic decision making and neural 
network processing. The OECD Group of Experts 
describes Al as including a 


machine-based system that can, for a given set 
of human-defined objectives, make predictions, 
recommendations or decisions influencing real 
or virtual environments. It uses machine and/ 
or human-based inputs to perceive real and/or 
virtual environments; abstract such perceptions 
into models (in an automated manner, eg, with 
Machine Learning or manually); and use model 
inference to formulate options for information 
or action. Al systems are designed to operate 
with varying levels of autonomy.”© 


The term Al is not a term of art and has been 
criticised by some experts for being vague or 
ambiguous. The Commission acknowledges and 
shares those concerns, but notes there is no ready 
alternative term that is widely understood. Where 
practical, this Report refers to the relevant specific 
technologies or techniques associated with Al, such 
as algorithms, mathematical formulae, computer 
code, machine learning, neural networks and 
automation. 


(b) Al-informed decision making 


‘Al-informed decision making’ includes a decision, 
or decision-making process, where Al is a material 
factor in the decision, and where the decision 

has a legal or similarly significant effect for an 
individual. The phrase ‘legal or similarly significant 
effect’ is used in the European Union's General Data 
Protection Regulation (GDPR).”” 


(c) Digital Communication Technologies 


Part D of this Report focuses on ‘Digital 
Communication Technologies’, which refers to 
technologies associated with information and 
communication technology (ICT); connected devices 
and the Internet of Things (loT); virtual reality (VR); 
and augmented reality (AR). These technologies are 
used in goods, services and facilities primarily for 
communication, and often have a user interface as 
a means of communicating information (eg, a voice- 
operated home automation system). 


2.5 Project process 


The Commission established an Expert Reference 
Group for this Project.2® Constituted by a diverse 
group of leading experts from technology, law, and 
science and academia, the Expert Reference Group 
advised the Commission on the Project's strategy, 
consultation process and recommendations. 


Four major partner organisations have supported 
this Project—the Australian Government 
Department of Foreign Affairs and Trade; Herbert 
Smith Freehills; LexisNexis; and the University of 
Technology Sydney. Project Partners contribute 
expertise and some financial and in-kind resources. 


Public consultation included: 


e roundtables with participants from industry, 
civil society, academia and government 


e inviting and receiving written submissions 
from stakeholders, especially views and 
input from people who are particularly 
affected by new technologies 


e interviews with experts and key decision 
makers 


* engaging with national review and reform 
processes on new technologies. 
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2021 


2020 


2019 


2018 


bo bbe 


38 recommendations to promote 
human rights in an era of new 
technologies 


How to address algorithmic bias 


The Commission’s preliminary 
views 


What kind of leadership is 
needed in Australia to promote 
responsible innovation? 


10 questions about human rights 
& new technologies 


Project launch with 450 
attendees, international and 
national expert presenters 


Engagement Highlights 


chit 


I 
725 


submissions received consultation participants 


conference presentations 
from the Human Rights 
Commissioner 


national survey 
participants 


2. Executive summary 


2.6 Outcomes 


The overall effect of this Report will be to foster 

a deeper understanding of the human rights 
implications for Australia of new and emerging 
technologies such as Al. By adopting a human 
rights approach, our country is well placed to seize 
the moment: to harness technology in a way that 
delivers what Australians want and need, not what 
they fear. 


The specific outcomes of this Project include: 


a better understanding of opportunities 
and threats posed by new and emerging 
technologies in Australia, especially for our 
human rights 


addressing the views of stakeholders from 
industry, government, civil society and 
academia, as well as leading experts from 
around the world 


a focus on responsible innovation in the 
development of Australia’s national strategy 
on new and emerging technologies 


a template for reform that will enhance 
accountability and human rights 
protections in Al-informed decision making 
by government and the private sector 


e key ways to support government and 
the private sector in using Al safely and 
accountably 


understanding the experience of people 
with disability in accessing goods, 
services and facilities that use Digital 
Communication Technologies 


reform that would assist in enhancing 
accessibility of Digital Communication 
Technologies. 


International human rights law sets out 
globally-accepted legal principles that 


uphold the dignity of all people. As a 
liberal democracy, Australia should place 
human rights at the centre of its approach 
to technology, with a view to promoting 
fairness, equality and accountability in 
the use and development of all new and 
emerging technologies. 
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PART A: NATIONAL STRATEGY ON 
NEW AND EMERGING TECHNOLOGIES 


3. National strategy on new and 
emerging technologies 


3.1 Summary 


This chapter addresses the question: what should be Australia’s overarching approach to 
new and emerging technologies? 


The Department of Prime Minister and Cabinet is currently leading a process, on behalf 
of the Australian Government, to create the Digital Australia Strategy.” This presents an 
excellent opportunity to articulate the key, big-picture elements of how Australia will 
respond to the rise of new and emerging technologies, such as artificial intelligence (Al). 


The Commission urges the Australian Government to embrace technological innovation 
that holds true to our liberal democratic values. This means putting human rights at the 
centre of how Australia approaches new and emerging technologies. 


The Commission recommends that the Digital Australia Strategy promote responsible 

innovation and human rights through measures including regulation, investment and 
education. This will help foster a firm foundation of public trust in new and emerging 
technologies that are used in Australia. 


The approach outlined in this chapter provides the foundation for the Commission's 
recommendations for law and other reform throughout this Report. 
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3.2 The need for a national strategy 


RECOMMENDATION 1: The Digital 
Australia Strategy, which is currently being 
developed by the Australian Government 
Department of the Prime Minister 

and Cabinet, should set Australia’s 
national strategy for new and emerging 
technologies. The Digital Australia 
Strategy should promote responsible 
innovation through: 


(a) effective regulation—including law, 
co-regulation and self-regulation— 
that upholds human rights in 

the development and use of new 
technologies 


(b) the development of a community- 
wide action plan on education, 
training and capacity building 
regarding the human rights 
implications of new and emerging 
technologies 


(c) funding and investment for 
responsible innovation that complies 
with human rights 


(d) practical measures to achieve the 
Strategy's aims, including through 
the establishment of an Al Safety 
Commissioner (see Recommendation 
22). 


Stakeholders across government, industry, 
academia and civil society overwhelmingly 
supported the development of an Australian 
national strategy on new and emerging 
technologies.*° 
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More than 25 countries and intergovernmental 
organisations have developed national or regional 
strategies for Al or new and emerging technologies, 
with many more in the process of development.*" 
The primary purpose of these strategies is to set 
overarching priorities, with a view to harnessing the 
economic and social benefits of Al, and addressing 
the most serious risks.*4 


National strategies generally contain objectives and 
action areas across a range of domains, including: 
research, talent and the future of work; private 

and public sector governance; regulation and 
ethics; infrastructure, data and foreign policy; and 
inclusion and stakeholder engagement. By setting 
clear government policy on funding, regulation and 
education, such strategies can: 


help the private sector to set its own 
planning and investment priorities 


guide government use of new technology, 
including in administrative decision making 


build community trust that protections are 
in place to address risks or harm 


contribute to the international development 
of policy guiding the development and use 
of technologies, and help identify common 
values and norms across governments.*4 


By way of illustration, the German national strategy 
on Al, Al made in Germany, has three overarching 
objectives that aim to promote technological 
innovation and the good of society. Germany 

seeks to be recognised as a developer and user 

of excellent Al ‘in all parts of society in order 

to achieve tangible progress in society in the 
interest of its citizens’.*° Similarly, the respective Al 
strategies of Canada and France set out overarching 
objectives aligned with responsible innovation and 
acknowledge the risk of harms associated with Al. 


The German national strategy acknowledges the 
tension that can arise between effective rights 
protections and the pursuit of investment and 
growth, and sets out a range of measures to help 
guide regulatory reform needed to protect rights, 
and at the same time fund the development of 
innovative applications that respect rights.?” 


While not a national strategy as such, the Executive 
Order on Al issued by the United States President 
in 2019 sets national priorities regarding research, 
innovation and commerce in Al in that country. 
Those priorities include advancing freedom, human 
rights protection, individual dignity and the rule 

of law.” Similarly, the Global Partnership on Al, of 
which Australia is a founding member, seeks to 


support the responsible and human-centric 
development and use of Al in a manner 
consistent with human rights, fundamental 
freedoms, and our shared democratic values.?9 


Good national or regional strategies on Al and other 
new technologies tend to have some common 
elements: 


they incorporate international human 
rights standards, including practical 

steps to ensure protection of rights with 
accountability and grievance mechanisms 


they promote human rights training and 
education for designers and developers of 
new technology 


they include measures for oversight, policy 
development and monitoring, such as the 
‘Al Observatory’ in A/ made in Germany“ 


they have whole-of-government 
approaches to ensure consistency across 
agencies and implementation 


they focus on present and future impacts of 
technologies on society. 


As anumber of similar nations have recognised, 
the creation of an overarching national strategy for 
Australia could be an effective way of responding 
to the pace of technological development and 
addressing the risks and benefits. An industry 
stakeholder submitted: 


governments are moving ahead with such 
strategies that aim at both creating excellence 
through economic investment as well as a 
robust regulatory framework that can set 

global standards and ensure trust. Australia is 
well-placed to compete in this evolving global 
landscape in order to promote innovation, 
economic growth and prosperity for its citizens.” 


This also presents an opportunity to put human 
rights at the fore. A leading North American 
technology company submitted that national 
strategies can 


embed a human rights-based approach into 
industry policy, for instance through the creation 
of tailored direct spending programs to help 
ensure that the design and technological 
foundations of right-respecting Al, such as 
transparency, explainability and accountability, 
are firmly established in key sectors.*? 


It is widely argued that a national strategy should 
be drafted broadly enough to account for rapid and 
continual development in technologies and how 
they are used.* Stakeholders submitted that this 
could include: 


e existing technologies, because new 
and emerging applications of existing 
technologies may engage human rights*° 


emerging technologies* 


the interoperability of technologies, given 
that ‘a technology may be benign, but in 
tandem with others may represent new 
risks and opportunities” 


reference to specific technologies that are 
known to engage human rights, such as 
drones, which use Al for facial recognition 
in public and private spheres,*#® or medical 
and health technologies.*° 


PwC submitted that an Australian national strategy 
should integrate approaches to technology 
pioneered by Aboriginal and Torres Strait Islander 
peoples to ensure that regulation and policy are 
inclusive.°° 
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3.3 Digital Australia Strategy: an 
opportunity for an overarching 
approach 


In addition to the ongoing process to develop the 
Digital Australia Strategy, the Australian Government 
has a number of processes underway, or 
completed, that contribute to Australia’s strategic 
approach to new and emerging technologies. These 
include: 


the Digital Economy Strategy, Al Action 
Plan, Al Ethics Framework and Al 
Technology Roadmap by the Department 
of Infrastructure, Science, Energy and 
Resources (DISER)?! 


the review of the Privacy Act 1988 (Cth) by 
the Attorney-General’s Department 


the Digital Transformation Agency's 
Strategy 2018-20255 


the cyber affairs and critical technology 
work and strategies of the Department of 
Foreign Affairs and Trade.** 


Stakeholders noted the importance of coordination, 
complementarity and alignment across these 
initiatives. For example, the Australian Industry 
Group urged better ‘coordination between the 
various agencies around policy issues that arise 
from new and emerging technologies’.*° 


The Government has recognised this need through 
the establishment of the Department of the Prime 
Minister and Cabinet's Digital Technology Taskforce. 
This Taskforce is responsible for developing the 
Digital Australia Strategy, which will outline a vision 
to ensure Australia is a leading digital economy and 
society by 2030.56 


In February 2021, following informal consultation, 
the Taskforce identified five overarching themes for 
the Strategy and invited public feedback.” The five 
themes are: 


1. moving more businesses to the digital 
frontier 

. a digitally capable and inclusive Australia 

. building digital trust 

. digital-first Government 

. lifting sectors through digital technology. 


UAWN 


These themes suggest that the Digital Australia 
Strategy is well placed to articulate a broad, national 
vision—one that also draws together the key 
existing strands of policy and law related to new 
and emerging technology in Australia. 


The Taskforce has invited comment on the 
overarching themes and the development of 

the Digital Australia Strategy more generally.” 

A significant number of stakeholders in the 
Commission's Project identified the importance 
of consultation with government, industry and 
civil society in developing a national strategy," 
especially those who are more likely to experience 
human rights violations from new technologies. 


The Allens Hub for Technology, Law and Innovation 
at UNSW emphasised the need for expert legal and 
technical input to develop a national approach: 


Legal and regulatory frameworks will need 

to be adjusted to resolve uncertainties in the 
application of existing rules, ensure that rules 
are not over- or under- inclusive with respect 

to activities facilitated by new technologies and 
make changes where existing laws or regulatory 
approaches are no longer justified, no longer 
cost-effective, or no longer applicable.” 


3.4 Elements of an Australian 
national strategy 


An Australian national strategy should include four 
primary aims: 
* to uphold human rights 


* to establish Australia’s priorities in 
regulating the development and use of new 
technologies 


to build a firm foundation of public trust in 
new technologies 


to develop an Australian plan for education, 
training and capacity building that responds 
to the exponential growth in new and 
emerging technologies. 


The Commission recommends that these elements 
be included in the Digital Australia Strategy. 


(a) Uphold human rights and promote 
responsible innovation 


Putting human rights at the centre of the Digital 
Australia Strategy would have several benefits. 


First, international law requires the Australian 
Government to take active steps to promote and 
protect human rights, including in respect of new 
and emerging technologies. Companies and other 
non-government bodies also are obliged to respect 
human rights under the United Nations (UN) 
Guiding Principles on Business and Human Rights.° 


Many countries have adopted national Al strategies, 
but few set out how human rights should be 
promoted, protected and respected in the use 

of Al.® For effective human rights protection, a 
national strategy must set out a clear, practical 
framework for action. It should set out specific 
steps to improve human rights protections, as well 
as effective oversight and dispute resolution.” For 
example, Germany's national strategy states that 
the use of Al must adhere to fundamental rights; 
and requires Germany's federal government to 
address any gaps in protection.® 


Secondly, international human rights law applies 
globally, and so provides a ready-made basis 

for international cooperation—something that 

is especially important in respect of new and 
emerging technologies. Placing human rights at the 
centre of the Digital Australia Strategy would reflect 
Australia’s commitment to be a leading rights- 
respecting state in this area. 


Thirdly, a clear commitment to human rights 

is a principled basis from which to pursue the 
opportunities presented by new and emerging 
technologies. That is, it would enable Australia to 
advance economic development goals in this area 
without jeopardising Australians’ basic rights. 


Finally, a human rights framework allows for 

the inclusion of ethical approaches to support 
human rights compliant development and use of 
technology. Ethical frameworks that uphold human 
rights can complement enforceable human rights 
protections, but they cannot replace them. 


Stakeholders noted the simultaneous benefits 
and risks of rapid technological development.® 
A national strategy could address both. For 
example, the Australian Council of Learned 
Academies (ACOLA) submitted: 


A national strategy that allows areas of major 
opportunity to be established while ensuring 
the range of social, ethical and legal challenges 
are embraced and held as core values for 
implementation would help support the safe, 
responsible and strategic development of 
emerging technologies.” 
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Striking a balance between the economic 
opportunities associated with innovation and 
safeguarding against harm is clearly important,® 
even more so in light of the COVID-19 pandemic. 
The NSW Bar Association submitted that a national 
strategy could 


assist with balancing interests to the extent 
these might come into conflict, such as balancing 
the paramount need to protect the community's 
health and safety with not impermissibly 
infringing upon individuals’ rights to privacy, 
freedom of movement and other rights, and 
other economic considerations and rights.°° 


Similarly, the Australian Red Cross submitted that 
new technologies are 


enabling better humanitarian approaches and 
solutions and enhancing our ability to deploy the 
right kind of response at the right time to where 
it is needed most. However, these tools and 
systems are also introducing a host of potential 
harms by exposing people and communities 

to new forms of intrusion, insecurity and 
inequality.”° 
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(b) Establishing Australia’s regulatory 
priorities 


Many new technologies are developed and used 
in ways that engage human rights and warrant an 
appropriate regulatory response. This, in turn, will 
depend on what is being regulated. 


A particular technology may be used in several 
different ways, engaging human rights to a greater 
or lesser extent depending on the specific context.” 
As a general rule, regulation should target the 
outcome of the use of that technology. Few types of 
technology require regulation targeted specifically 
at the technology itself.”2 


It is also important to consider what is the most 
appropriate type of regulation in the particular 
circumstance. For example, self- and co-regulation 
has some advantages where there is rapid growth 
and change in an area of technology. Those 
advantages include the speed of establishment 
and revision, the incorporation of sector-specific 
knowledge, and encouragement to stakeholders to 
understand and accept the regulatory process.” 


Effective national regulation should contribute to 
the other key aims of the Digital Australia Strategy: to 
uphold and protect human rights; and instil trust in 
the public about how new technologies are used in 
Australia. Three key principles should apply: 


1. Regulation should protect human rights. 
As a first principle, all regulation should 
be guided by Australia’s obligations under 
international law to protect human rights. 


2. The law should be clear and enforceable. 
Australian law should set clear rules 
regarding the design, development and 
use of new technologies, with a view to 
promoting human rights and other aims 
consistent with our liberal democracy. 
The broader regulatory framework 
should contain effective remedies and 
enforcement mechanisms. Australia’s 
lawmakers should fill any gaps necessary 
to achieve these regulatory aims. 


3. Co-regulation and self-regulation should 
support human rights compliant, ethical 
decision making. The law cannot, and 
should not, address every social implication 
of new and emerging technologies. 

Good co- and self-regulation—through 

professional codes, design guidelines and 
impact assessments—can assist in making 
sound, human rights-compliant decisions. 


There was strong support among stakeholders 
for the view that regulation—involving a smart 
mix of adaptive and anticipatory law, co- and self- 
regulation—should uphold human rights through 
enforceable and clear rules.”4 


Stakeholders noted several benefits to this 
approach: 


* co- and self-regulation can be a useful 
adjunct to legislation, which tends to 
take more time to implement, where 
there is a need to respond to fast-moving 
technological change” 


self-regulation alone does not afford 
adequate human rights protections 
generally, nor does it guarantee 
accountability and the right to a remedy” 


different types of technologies require 
tailored regulatory approaches.’ 


Some stakeholders referred to the importance of 

a human rights approach,” with several reiterating 
that the most effective way to protect human rights 
in the development and use of new technology 
would be through a national bill of rights or charter 
of human rights.’ 


Concerns about the use of new technologies in the 
corporate sector have encouraged some companies 
to be proactive in ensuring their products and 
services do not cause harm. For example, as 
discussed in Chapter 9, some major technology 
companies have set limits on the use of facial 
recognition for this reason. 


Presently, such initiatives are largely driven by 
individual companies. There would be benefit 

in the Australian Government building on this 
momentum—ideally working cooperatively with 
other states—to foster a more coordinated 
approach to self- and co-regulation. For instance, 
the use of self-certification and ‘trustmarks’, 
discussed in greater detail in Chapter 7, could signal 
to consumers what sorts of tech-powered products 
and services are likely to be designed in ways that 
protect users’ human rights. Similarly, government 
can use its own procurement rules and processes 
as a way of encouraging the development and use 
of human rights respectful technology. 


Positive innovation, which delivers economic 

and broader social benefits, can flourish in 
environments where there is a clear understanding 
of how human rights need to be protected and 

the parameters in which they can operate.®° 
Additionally, the public’s trust in the design, 
development and use of new technologies should 
improve as regulatory parameters are defined and 
applied. 


Stakeholders noted the importance of ensuring that 
existing law is understood and applied correctly to 
new technology.®' Microsoft submitted: 


A multi-faceted regulatory framework is 
appropriate, involving the application of 
existing laws as well as ‘soft law’ co-regulation 
and self-regulation mechanisms such as 
international standards, industry best practices 
and guidelines, and international governance 
procedures—to ensure the development and 
use of Al is responsible and aligned with human- 
centred values.*® 


Some stakeholders noted a tension between 
regulation and innovation.® The Australia Industry 
Group submitted: 


there should be careful consideration of any new 
forms of regulation against global best practice 
approaches and the extent of Al industry 
overseas. A similar argument could be extended 
to other new and emerging technologies as 
well.®4 
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The Allens Hub for Technology, Law and Innovation 
at UNSW submitted that effective regulation should 


focus on ensuring that legal and regulatory 
frameworks are well-adapted to an evolving 
socio-technical landscape, curating it in ways 
that ensure alignment with human rights and 
other values of importance to Australians.®> 


Some stakeholders noted the challenges to 
effective national regulation in a global economy, 
observing that Australia is a net importer of 
technology, and we need to ensure comparable 
human rights protections with trading partners.* 
Telstra submitted: 


it would be useful for any proposed Australian 
approach to be broadly aligned with what is 
happening in other developed countries, so 

as to facilitate technology development and 
allow Australia to take advantage of Al-enabled 
systems that are supplied on a global scale 

and thus minimise the cost for users to comply 
specifically, and potentially additionally, with an 
Australian framework.®” 


(c) Public trust 


Australia’s A/ Action Plan rightly observes that 

‘to spur broader adoption of Al in ways which 
contribute to Australia’s economic prosperity, 
security and unique way of life, we need to maintain 
public trust in Al’.8° Public trust is essential for 
Australia to harness the opportunities presented by 
new and emerging technologies. 


Public trust in some new technologies is low, with 
only one in three respondents to a national survey 
in 2020 willing to trust AI. Yet Australians have also 
expressed high levels of support for the use of Al 
to address social, humanitarian and environmental 
challenges.” This suggests a general recognition of 
the potential for new technologies, and especially 
Al, to deliver benefits for the community, but 

a concern that the current products, services 

and facilities that use these technologies are 
inadequate. 


Other jurisdictions are also grappling with this issue 
and see effective regulation, grounded in human 
rights and the rule of law, as key to building trust. 
The European Commission states: 


Given the major impact that Al can have on our 
society and the need to build trust, it is vital 
that European Al is grounded in our values and 
fundamental rights such as human dignity and 
privacy protection.” 


The United States President’s Executive Order on 
Promoting the Use of Trustworthy Al in the Federal 
Government aims to 


foster public trust and confidence in the use 
of Al, protect our Nation's values, and ensure 
that the use of Al remains consistent with all 
applicable laws, including those related to 
privacy, civil rights, and civil liberties.% 


It was clear from the Commission's consultations 
that enhancing the reliability and safety of new 
technologies, including by addressing risks of harm, 
would help build a solid foundation of community 
trust in the development and use of those 
technologies.” Robust human rights protections 
are considered essential for that public trust.’ 


The Law Council of Australia submitted that 

the protection of human rights ‘especially [for] 
vulnerable and disadvantaged groups is critical 
to building enduring public trust in technology, 
noting the importance of practical steps for that 
protection’. The Australian Academy of Science 
noted that ‘building public trust should be about 
building trustworthy systems’.°° 


Australians see a critical role for government in 
this area. The majority of participants in a national 
survey in 2020 expected government oversight of 
the development and use of AI.” Another national 
survey, commissioned by this Project, found that 
74%-78% of respondents would have ‘a little more’ 
or ‘a lot more’ trust in an automated decision if 
oversight measures such as stronger human rights 
protections, human checks and limitations on 
personal information sharing were implemented.” 


(d) Education, training and capacity building 


There is a need for all parts of Australian society— 
government, industry, civil society and the 
community at large—to understand how new 
technologies, including Al, affect their professional 
and personal lives. 


A number of jurisdictions recognise the critical 
role of education, training and capacity building 
in managing digital transformation. The European 
Commission's Digital Education Action Plan 2021- 
2027, for example, prioritises education and 
training in digital transformation.” 


The Australian Government has also recognised 
the importance of such education and training 

in a number of its official policy statements. For 
example, the Al Action Plan sets out four streams 
of action to maximise the benefits of Al for all 
Australians, with one stream being for Australians 
to ‘have the skills and capability to benefit from 


and participate in an Al-driven society’. A possible 
focus action for this stream, currently subject 

to consultation, is to ‘ensure Australians have 
foundational Al education and skills’.'° 


Stakeholders generally supported the idea of 
education and training being included in any 
national strategy in this area, and that this be 
tailored to the particular skills and knowledge 
needs of different parts of the Australian 
community.” The Commission did not receive any 
submissions that opposed this idea. 


Stakeholders tended to see Al as the most pressing 
current area of technological change. Some 
recognised the importance of education to equip 
the community at large to respond to the rise 

of Al in an ever-increasing range of contexts,'” 

and better enable people to protect their rights, 
especially when Al is used in decision making that 
affects individuals.'° This, in turn, could contribute 
to community trust in Al. 
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Similarly, in June 2020, the NSW Department of 
Education published an education innovation 
strategy, which set out the aim to equip students 
with the necessary skills and knowledge to 
‘successfully navigate a more complex world’, 
including ensuring students are able ‘to understand 
and navigate the moral and ethical impacts’ that Al 
will have on our lives, ‘such as the use of big data in 
real world decision making’.'“ 


In addition, there is a recognised need for targeted 
workforce education. For example, the Artificial 
Intelligence Roadmap, produced by CSIRO’s Data 
61, identified the need to build an Al specialist 
workforce,’ and the Australian Government 
established the ‘Digital Skills Organisation’ in mid- 
2020 to ‘provide an innovative and world-leading 
approach to training and employing people in 
digital skills in Australia’.1° 


Several stakeholders made specific suggestions in 
this area, which included: 


* targeting education and training on 
human rights and ethics towards those 
professional and other groups, such as 
data scientists, engineers and so on, who 
are most directly involved in research and 
development involving Al and related new 
and emerging technologies! 


the need to involve universities directly in 
curriculum development!” 


prioritising capacity building training for 
workers who use Al and related technology 
to make decisions that affect people's legal 
and similarly significant rights!” 


collaborating across the private and 
public sectors to ensure cooperation 
and best practices,"? and drawing on 
existing education initiatives in industry, 
government and internationally" 


aligning education with the most relevant 
areas of regulation." 


Accordingly, the Commission considers that the 
Digital Australia Strategy should prioritise education, 
training and capacity building that addresses this 
need in three ways. 


First, almost everyone in Australia uses tech- 
powered products and services, and is subject 

to their use, including in decision making by 
government and the private sector. However, 
recent research indicates that people feel ill- 
equipped to engage in a meaningful way with 
new technologies such as Al. For example, the 
majority of survey participants in one poll ‘consider 
themselves to have little knowledge of Al',113 and 
another survey revealed that 86% of respondents 
want to know more about Al.'"4 


This reinforces a need for awareness raising and 
broad upskilling of the Australian community 

at large, to ensure that people understand how 
they are affected by these technologies, the risks 
involved and what steps they can take to benefit 
from the use of new technologies without suffering 
harm. 


Secondly, at present, there is little targeted training 
regarding the human rights implications of Al for 
those who are responsible for developing and using 
Al. This can contribute to a failure to perceive or 
address risks of harm. The Digital Australia Strategy 
should also meet this need. 


People responsible for designing, developing, using 
and overseeing the use of new technologies need 

a deeper, more sophisticated understanding of the 
technologies themselves and their human rights 
implications. 


Part B of this Report considers the use of Al to make 
decisions that have legal or similarly significant 
effects. Education, training and capacity building 
should be targeted especially towards people 

who are responsible for developing and using Al- 
informed decision-making systems, to ensure that 
those systems operate accurately, fairly and in ways 
that uphold human rights. 


Thirdly, there is a need to build the capacity of 
regulatory and oversight bodies. As discussed 

in Part C, they would benefit from a deeper 
understanding of how best to fulfil their functions 
in an environment that is increasingly defined by 
the presence of new technologies like Al. This is 
not a call for every regulator to be an expert in 
the science and engineering of Al and other new 
technologies. 


Rather, they do need to understand how these 
technologies operate and specifically how they 
affect their work. They also need access to technical 
expertise to support their regulatory activity. 


Equipping these regulatory bodies for this new era 
should involve the following activities: 


* targeted education and training for decision 
makers and leaders in these bodies 


* resourcing these bodies to better 
understand and explain how existing rules 
apply in the context of new technologies 


* encouraging these bodies to promote 
human rights-compliant behaviour in the 
sectors in which they operate. 


The Commission recommends in Part C the 
establishment of an Al Safety Commissioner 

to provide guidance on the development, use 
and regulation of Al—with a particular focus on 
regulatory and governance bodies."'° 


To draw these three areas of education and training 
together, the Commission recommends that the 
Government develop a comprehensive plan for 
Australia regarding education and training on new 
and emerging technologies and human rights, as 

an action arising under the Digital Australia Strategy. 
This education plan should respond to the differing 
needs across the Australian community, which can 
be broadly divided into the following groups. 
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Group 


General public 


Objective and outcome 


All individuals will need a basic understanding of Al 

and how it is being used. Training should enable the 
community to make informed choices about how 

they interact with organisations that use Al in decision 
making, and the accountability mechanisms available 
to ensure that decisions that affect them are lawful and 
respect their human rights. 


Leadership to develop 
educational modules 


Government 
Schools 
Civil society 


Industry bodies with a public 
education function 


Decision makers 
who rely on Al 
datapoints 


Decision makers who use Al datapoints to make 
decisions need sufficient knowledge of the reliability of 
various types of Al datapoints to make good decisions. 
This is likely to include an expanding group of decision 
makers, such as judicial officers, insurance brokers and 
police officers. 


Employers of decision 
makers 


Professional regulatory and 
organisational bodies that 
are involved in professional 
development 


Professionals 
requiring highly- 
specialised 
knowledge 


Designers and developers of Al-informed decision- 
making systems need to be able to predict the likely 
impact of their work, including how to incorporate 
human rights in the design process. The accreditation 
or certification of professionals who design Al should be 
considered. 


Regulators, policy makers and those who commission Al- 


informed decision-making systems need to understand 
how Al operates and its likely impact on human rights in 
the context in which systems will be deployed. 


Government 
University curriculum 


Professional regulatory and 
organisational bodies 


Technology companies 


3.5 Conclusion: Australia’s national 
strategy 


The Commission supports the Australian 
Government's process to develop the Digital 
Australia Strategy. This can provide the vehicle 
for Australia to set out its national strategy on 
the development and use of new and emerging 
technologies. 


The Commission recommends that the Department 
of the Prime Minister and Cabinet's Digital 
Technology Taskforce use the Digital Australia 
Strategy to articulate how Australia’s various 
initiatives on Al and other new technologies 
interconnect, how they complement each other, 
and ultimately how they contribute to a coherent 
and positive vision for how Australia will approach 
new and emerging technologies. 


The Commission commends the Department 
of Prime Minister and Cabinet for providing an 
opportunity for public input in the development 
of the Digital Australia Strategy. The Commission 
understands that the Strategy is intended to 
bea ‘living document, adapted over time as 

the technology context continues to evolve. 


The Commission thus recommends ongoing 
consultation with all relevant stakeholders, 
including industry and civil society. Special effort 
should be made to obtain the views of people who 
are likely to be particularly affected by the use of 
new and emerging technologies. 


The Digital Australia Strategy should, in principle, 
cover technologies that are known, emerging, and 
unforeseen, to ensure human rights protections 
into the future. A scoping exercise involving 
technical and legal experts could help establish 
how some technological fields (eg, health, aviation, 
biomedical) are treated in the Strategy to ensure 
complementarity with other oversight or regulatory 
processes. 


The Commission concludes that the Digital Australia 
Strategy should include four key objectives: 


* to uphold human rights and promote 
responsible innovation 

* to guide effective regulation of the 
development and use of new technologies 

* to build a firm foundation of public trust in 
new technologies 

* to develop an Australian plan for education, 
training and capacity building to respond to 
new and emerging technologies. 
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4. Using artificial intelligence to 
make decisions 


4.1 Summary 


The use of artificial intelligence (Al) is changing how important decisions are made by 
government and the private sector. This has significant human rights implications. 


Some uses of Al do not meaningfully engage anyone's human, legal or other rights. For 
example, sophisticated Al is commonly used in computer games, which are developed 
purely for entertainment. 


This Project has focused on the use of Al to make decisions that have legal or similarly 
significant effects on individuals. Such decisions are likely also to engage human rights in 
a more profound way. 


It is difficult to overstate Al's potential, or indeed the impact it is already starting to 
have. Australian governments are seeking to harness the potential of Al to drive better, 
more efficient services, products and more.""® At the same time, there is growing public 
concern that Al can be used in ways that cause harm.'"” 


Over the past two years, the Commission has consulted the community regarding the 
human rights implications of Al, especially where it is used in decision making that 
affects people’s legal and similarly significant rights. 


In Part B of this Final Report, the Commission makes recommendations to ensure human 
rights are protected where Al is used in decision making—something that requires 
accountability. In particular, the Commission has concluded that Al-informed decision 
making should be lawful, transparent, explainable, used responsibly, and subject to 
appropriate human oversight, review and intervention. 


This first chapter of Part B explores the critical importance of accountability in upholding 
human rights in Al-informed decision making. It sets out principles to promote and 
enhance accountability in this area. 


The remaining chapters in Part B apply these principles: 


Chapter 5 considers legal accountability in respect of Al-informed administrative 
decision making—that is, the use of Al by government to make decisions 


Chapter 6 considers legal accountability in respect of Al-informed decision 
making by corporations and other non-government entities 


Chapter 7 addresses the role of co- and self-regulation to incentivise better 
human rights protections in the development and use of Al-informed decision- 
making systems 


Chapter 8 explores the problem of ‘algorithmic bias’ in Al-informed decision 
making, and how it can lead to unfairness and discrimination 


Chapter 9 focuses on the use of biometric surveillance, including through facial 
recognition technology. 


Human Rights and Technology Final Report * 2021* 35 


36 


Critical 


risks 


Biometric 
surveillance & 
facial recognition 


Algorithmic bias: 


addressing 
unfairness & 


discrimination 


Govt use of Al in 


administrative 
decisions 


Mandatory Al 
notification 


Explainable Al: 
preserving a right 
to reasons 


Mandatory human 
rights impact 
assessment 


Human review 


Promoting 


accountable Al 


Government Human rights Voluntary 

procurement by design standards, 
certification & 

regulatory 

sandboxes 


Private sector 


use of Al 


Encouraging 
explainable Al 


Legal liability & 
court oversight 


Encouraging 
human rights 
impact 
assessment 


Mandatory Al 
notification 


4.2 Al and terminology 


The term Al is widely used, especially in popular 
debate. However, Al does not have a precise, 
universally-accepted definition,'"® nor does it refer 
to a single technology. 


Most commonly, Al is understood as a cluster or 
‘constellation’ of technologies and techniques,"'® 
which include some forms of automation, machine 
learning, algorithmic decision making and neural 
network processing. The Organisation for Economic 
Cooperation and Development (OECD) Group of 
Experts describes Al as including a 


machine-based system that can, for a given set 

of human-defined objectives, make predictions, 
recommendations or decisions influencing real 

or virtual environments. It uses machine and/or 
human-based inputs to perceive real and/or virtual 
environments; abstract such perceptions into 
models (in an automated manner, eg with Machine 
Learning or manually); and use model inference 

to formulate options for information or action. 

Al systems are designed to operate with varying 
levels of autonomy. 


The Commission acknowledges valid criticism that 
the term Al can be ambiguous and misleading.” 
Given the term is so widely used, this Report refers 
to Al but, where practical, it refers to the relevant 
specific technologies or techniques associated 
with Al. In a similar way, the United Kingdom's 
Information Commissioner’s Office (UK ICO), for 
example, uses the umbrella term Al, because it ‘has 
become a mainstream way for organisations to 
refer to a range of technologies that mimic human 
thought." 


While the technologies and techniques associated 
with Al have long histories, their capabilities have 
advanced with unprecedented speed in recent 
years. There are several reasons for this, not least 
the growing availability of large datasets, increased 
computing power and new programming and data 
analysis techniques. '?? 


This Project has focused primarily on the sorts of 
Al that exist now and foreseeably—sometimes 
referred to as ‘narrow Al’.'*4 It has not specifically 
considered the implications of speculative new 
forms of Al that might arise in the future, such as 
‘artificial general intelligence’. 


Some technologies and techniques associated with 
Al can significantly engage human rights when used 
in decision making. These include: 


* algorithms, mathematical formulae and 
computer code, which are ‘designed and 
written by humans, carrying instructions to 
translate data into conclusions, information 
or outputs"? 


machine learning and neural networks, 
involving a computer program learning to 
undertake defined tasks using numerous 
examples in a dataset, detecting patterns in 
the examples. The computer program then 
uses those patterns to make inferences, 
predictions, recommendations or decisions 
for a new case or situation that was not 
included in the original data set.’7° This 
process can be: 


» unsupervised learning, where the 
system is fed a dataset that has not 
been classified or categorised, and it 
identifies clusters or groupings 


y 


supervised learning, where a system is 
fed a categorised or classified data set 
that it uses to learn how to complete a 
task as instructed, or 


y 


reinforcement learning, where a 
system takes action within a set 
environment and assesses whether 
that action achieves its goals, and it 
learns, through this process, which 
action to take to achieve those goals!” 


automation involving a computational 
system applying algorithms or other rules 
to particular fact scenarios in an automatic 
way. A decision-making system can be 
wholly automated, in which case it produces 
decisions without human involvement, 

or the system can produce inferences, 
predictions or recommendations, which 

a human will use to make a final decision. 


Human Rights and Technology Final Report * 2021 ° 37 


4. Using artificial intelligence to make decisions 


38 


4.3 Al-informed decision making 


(a) What is Al-informed decision making? 
The term ‘Al-informed decision making’ refers to: 


i. a decision or decision-making process, 


ii. that is materially assisted by the use of an 
Al echnology or technique, and 


iii. the decision or decision-making process has 
a legal, or similarly significant, effect for an 
individual. 


The purpose of this definition is to bring into focus 
some critical ways in which technology and decision 
making combine to create human rights and 
accountability issues that need close attention by 
policy and law makers. 


The Commission does not propose a statutory 
definition of Al-informed decision making. As a 
number of stakeholders observed, decisions are 
generally the proper subject of regulation, not the 
technology that is used to make those decisions.' 
Consequently, it would be problematic to define or 
create legal rights and obligations by reference to 
whether or not a decision involved the use of Al. 


(i) A decision or decision-making process 


A person's human rights can be affected by a 
decision, or by a decision-making process. That is, 
human rights might be engaged by an outcome, 
such as the rejection of a bank loan to an individual 
on the basis of their gender. Human rights may also 
be engaged by a process, such as the inclusion of 
race as a relevant weighted factor in an algorithmic 
tool determining the risk of reoffending. Both 
elements should be taken into account when 
seeking to safeguard human rights. 


An example of an Al-informed decision-making 
process is highlighted in Figure 1. 


Figure 1: Al-informed decision-making process‘? 
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(ii) Materially assisted by the use of Al 


There are many ways in which Al may be used in 
decision making. In some cases, the use of Al will 
be central to the decision-making process, and the 
ultimate decision. For others, Al will have only a 
trivial impact on the decision-making process. 


Sometimes it is clear that the use of Al was material 
in making a decision. Where, for example, all key 
elements of the decision-making process are 
automated, the use of Al is clearly material. 


In other cases, the materiality of Al in the decision- 
making process can be more difficult to gauge. 

For example, Al can be used to generate a data 
point that a human decision maker then relies on 
to make the ultimate decision. While generally this 
will involve a material use of Al, the specific context 
is important. If, say, the human decision maker 
types their decision using a sophisticated word- 
processing application that was developed using 
Al, and the application simply records the decision, 
this use of Al (via the word-processing application) 
would not be material in the relevant sense. 


Humans decide 
how to apply 
and use the 

outputs. 


(iii) Legal or similarly significant effect for an 
individual 


The Commission is primarily concerned about 
decision making that engages human rights, and 

so this Final Report focuses on decisions that have 

a legal or similarly significant effect on an individual. 


The phrase ‘legal or similarly significant effect’ is 
used in the EU's General Data Protection Regulation 
(GDPR).'2° The UK ICO states that a decision under 
the GDPR produces ‘legal effects’ if it affects an 
individual's legal status or legal rights, such as the 
ability to access a social security benefit. A decision 
that has a ‘similarly significant effect’ has an 
equivalent impact on an individual's circumstances, 
behaviour or choices, such as the automatic refusal 
of an online credit application.'*' 


The phrase ‘legal or similarly significant effect’ is 
therefore useful in delineating the categories of 
decision that are likely to engage human rights 
more broadly. 


Human Rights and Technology Final Report * 2021 ° 39 


4. Using artificial intelligence to make decisions 


Box 1: Approaches to defining Al for policy or regulatory purposes 


Governments, regulators and others have approached the protection of human rights in the 
context of Al in different ways. While some authorities overseas have chosen a precise definition 
of Al, others have adopted a risk- or context-based approach. 


Some bodies distinguish between different types of technology. For example: 


e The UK Information Commissioner's Office has observed that Al refers to ‘various methods 
for using a non-human system to learn from experience and imitate human intelligent 
behaviour’. It distinguishes between Al, machine learning, and big data analytics, noting the 
subtle differences between these terms.'*2 


* The Algorithm Charter for Aotearoa New Zealand focuses on algorithms, which include 
analytical tools ranging from decision trees and regression models to more complex 
systems using machine learning, such as neural networks, to make advanced calculations 
and predictions.” 


Some approaches focus on the approximation of human or intelligent thought. For example: 


* France's Al National Strategy states that Al ‘refers to a programme whose ambitious objective 
is to understand and reproduce human cognition; creating cognitive processes comparable 
to those found in human beings’.'34 


* Canada’s Directive on Automated Decision-making defines Al as ‘information technology that 
performs tasks that would ordinarily require biological brainpower to accomplish, such as 
making sense of spoken language, learning behaviours or solving problems’. An automated 
decision system is defined as ‘technology that assists or replaces the judgment of human 
decision-makers’.'?> 


Other bodies have adopted a contextual approach to focus regulatory objectives. The European 
Commission's Joint Research Centre, for example, has noted commonalities among the different 
definitions of Al adopted by government and industry: 


* Perception of the environment, including the consideration of real-world complexity 
* Information processing, including collection and interpretation of inputs 


e Decision making (including reasoning and learning), such as taking actions and performance 
of tasks with a certain level of autonomy 


* Achievement of specific goals, which is the ultimate purpose of Al systems.' 


Similarly, the work of the Australian National University’s 3A Institute on addressing the impact 
of Al focuses on ‘cyber-physical systems’, the core features of which 


are an ability to automatically sense the environment (drawing from loT connect datasets or creating 
new data through sensing technology), to infer something from this data, and to act upon that data in 
a way that has real and unmediated effect in the world.'%” 
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4.4 Al-informed decision making 
and human rights 


Al is increasingly being used in decision making 
in areas as diverse as criminal justice, advertising, 


recruitment, healthcare, policing and social services. 


While the use of Al can protect and promote human 
rights, such as by identifying and addressing 
instances of bias or prejudice that can be present 

in human decision making, the opposite can 

also be true, with Al also capable of reinforcing or 
exacerbating biases or prejudices. 


Drawing on the Commission's consultations and 
recent developments in science and law, this 
section considers some of the ways in which Al- 
informed decision making engages human rights. 
This includes both an impact on individual human 
rights, and how the use of Al is having a broader, 
systemic impact on groups and society itself. 


(a) Impact on human rights 


Until recently, public debate about Al and human 
rights focused almost exclusively on the right to 
privacy. However, the use of Al can affect a much 
broader range of civil and political rights, and 
economic, social and cultural rights. 


For example, in June 2020 the United Nations 

(UN) Special Rapporteur on contemporary forms 
of racism, racial discrimination, xenophobia and 
related intolerance noted the growing use of digital 
technologies, including Al, to ‘determin[e] everyday 
outcomes in employment, education, health care 
and criminal justice, which introduces the risk of 
systemised discrimination on an unprecedented 
scale.” 


The UN Special Rapporteur on the right to freedom 
of opinion and expression,"*! and the UN Special 
Rapporteur on extreme poverty and human 
rights,'42 have made similar reflections in their 
respective areas of focus. There is also growing 
academic and civil society work in this area.'® 


In this section, the Commission considers three 
areas of government decision making where the 
use of Al poses particular human rights risks. 
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(i) Automated government services The UN Committee on Economic, Social and 

Cultural Rights has concluded the right to social 
Governments in Australia and overseas are security imposes an obligation on governments 
increasingly considering or using Al in service to ensure that eligibility criteria for social security 
delivery. In September 2020, for example, the NSW benefits are ‘reasonable, proportionate and 
Government released an Al strategy to guide its transparent’. Further, any ‘withdrawal, reduction 
expansion of digital government service delivery." or suspension’ of social security benefits should 
However, there remain risks. As a Council of Europe be circumscribed and ‘based on grounds that are 
study observed, automation by government can reasonable, subject to due process, and provided 
reduce transparency and accountability, and for in national law’.15° 


increase the risk of arbitrary decisions.'45 


The use of Al, and especially automation, in 
delivering government services can engage human 
rights including the right to social security and an 
adequate standard of living,'*° the right to non- 
discrimination and equality,'*” and the right to an 
effective remedy.'48 


Case Study 1: 
Al used in social services 


Many stakeholders expressed strong views about the human rights implications of automation 

in the social security area. Particular concern was directed towards an Australian Government 
program of automated debt recovery in relation to social security payments. Some have called this 
program ‘Robodebt’."*! 


The automated debt recovery system used an algorithm to identify any discrepancies between 

an individual's declared income to the Australian Taxation Office, and the individual's income 
reported to Centrelink. Where a discrepancy was identified, this was treated as evidence of 
undeclared or under-reported income, and a debt notice was automatically generated and sent to 
the individual.'°? 


A parliamentary inquiry concluded that this process resulted in some inaccurate debt notices.'° 
As this debt recovery program related to welfare or social security payments, these errors 
disproportionately affected people who were already disadvantaged or vulnerable.'** Drawing on 
its casework experience, Victoria Legal Aid submitted that the scheme was poorly designed, unfair, 
and undermined public confidence in government decision making.'> 


On 29 May 2020, Services Australia announced all debts raised wholly or partly under the 
automated debt recovery system would be refunded; a total of $721 million across 470,000 debts 
would be refunded, affecting 373,000 people.'° 


In November 2020, the Prime Minister announced the Australian Government would pay a further 
$112 million in compensation payments as part of an agreement to settle a class action lodged 

in November 2019. The Government also agreed, as part of the settlement, to not pursue the 
payment of an additional $398 million in debts that had been wrongly raised.'%” 
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(ii) Risk assessment in criminal justice settings 


Data-driven risk assessment tools are used 
increasingly to predict the likelihood of future 
criminal behaviour. These tools are starting to 

be rolled out in a number of countries to assist 
decision making in the criminal justice system, 
including decisions regarding sentencing, bail and 
post-sentence restrictions on people assessed as 
being likely to commit further crime. 


The use of Al in the criminal justice system engages 
several human rights, including the right to equality 
and non-discrimination, the right to equality before 
the law, personal security and liberty, the right to 
privacy, the right to a fair and public hearing, the 
right to procedural fairness, and the presumption 
of innocence. '°8 


Where these risk assessment tools use machine 
learning, there is a significant risk of algorithmic 
bias, unless this problem has been effectively 
addressed through the design of the relevant 
tool.'9? A common problem is that a particular 
racial or ethnic group may have experienced 
disproportionate scrutiny by police over a long 
period of time, which in turn contributes to 
members of that group being convicted at a 
disproportionate rate for particular crimes. Where 
those statistics are used in training data to create 
risk assessment tools, these tools are likely to ‘learn’ 
that members of that group are more likely to 
commit crime. 


Even where this problem is identified, it can be 
difficult to address, because of a related problem: 
the risk assessment tool could rely on an apparently 
unrelated factor, which is, in reality, a close proxy 
for racial or ethnic origin. For example, it is common 
for particular areas or postcodes to be populated 
by people of a similar racial or ethnic background. 

If a risk assessment tool takes account of postcode, 
this could have the indirect consequence of 
considering race or ethnicity." Accordingly, while 
some risk assessment and policing tools give 

the appearance of neutrality, they can reinforce 
discriminatory outcomes at scale." 


There remains a live debate among experts about 
how to address these problems effectively. Yet, 
inthe meantime, governments in Australia and 
overseas are increasingly using these types of 
tools.'© 


In Australia, several stakeholders pointed to the 
disproportionately high number of Aboriginal and 
Torres Strait Islander peoples subject to the Suspect 
Target Management Program, an algorithm-based 
tool that NSW Police uses to identify individuals 

at risk of committing criminal offences.'°? While 
revisions have since been made to the Suspect 
Target Management Program, and it has been the 
subject of further review, the criticisms of at least 
the original version of this tool remain relevant.'™ 


In 2019, the Chicago Police Department 
decommissioned a predictive policing tool used 

to list and rank individuals at risk of committing 
violent crime.'® On reviewing the risk models, the 
City of Chicago Office of Inspector General found 
that the risk scores were unreliable as they were 
not updated regularly, and had poor data quality.’ 
The Inspector General also found that the tool 
risked entrenching discriminatory police practices 
and failed to reduce violence.'®” 


As a number of stakeholders observed, there is 
often little publicly available information regarding 
how these risk assessment tools operate, and how 
they are used to make decisions.'® This can make 
it difficult, if not impossible, for individuals subject 
to these tools to challenge decisions about them, 
which can limit human rights including the rights to 
liberty and freedom of movement.'% 


In 2015, the New South Wales Civil and 
Administrative Tribunal accepted a request from a 
state police force not to release information relating 
to one of its risk assessment tools.'”° In other cases, 
legal challenges have led to greater information 
being provided. In the United States, for 

example, a legal challenge to the New York Police 
Department's refusal to provide records regarding 
the testing of predictive policing technologies led to 
the disclosure of documents that shed some light 
on some commercial software and algorithms used 
for predictive policing.” 


Some who use these Al-informed risk assessment 
tools—especially from within the judiciary— 

have started to question their reliability." Such 
concerns have prompted calls to halt the use of 
these types of risk assessment tools in the criminal 
justice system, at least until effective human rights 
and accountability mechanisms are in place. 
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For example, a recent report by the US-based 
Partnership on Al concluded: 


Using risk assessment tools to make fair decisions 
about human liberty would require solving deep 
ethical, technical, and statistical challenges, 
including ensuring that the tools are designed and 
built to mitigate bias at both the model and data 
layers, and that proper protocols are in place to 
promote transparency and accountability. The 
tools currently available and under consideration 
for widespread use suffer from several of these 
failures.'73 


(iii) Risk assessment and automation in 
immigration settings 


Al has been considered and used for immigration 
decision making in Australia, the United States, 
Canada, the United Kingdom and elsewhere. 

The use of risk assessments, predictive analysis, 
automated and algorithmic decision-making 
systems in an immigration context engages a range 
of human rights, including the rights to equality and 
non-discrimination, and the rights to life, liberty and 
security of the person. In some situations, it can 
also engage citizenship rights and the freedoms of 
expression, religion, association and movement. 


In some jurisdictions, automated decision making 
is starting to be used to assist in determining an 
individual's claim for asylum, and can augment 

or even supplant human discretion. This engages 
obligations under the UN Refugee Convention.'” 


For example, the Canadian Government has used 
predictive analysis and automated tools to process 
immigration or visa applications in a way that 
engages a range of human rights, including the 
right to non-discrimination, as well as fundamental 
rules of procedural fairness.'”> Immigration, 
Refugees and Citizenship Canada developed a 
‘predictive analytics’ system to automate some 
evaluation of applications." The government 
department is using a form of automated system 
to triage applications into ‘simple’ and ‘complex’ 
streams.'”” Applications classified as ‘complex’ are 
then flagged for further human review.'” 


The UK Home Office recently abandoned an 
algorithmic decision-making tool used to grade 
entry visa applications. The tool assigned a risk 
rating to applicants, which then played a major 
role in the outcome of the application. A result 
was greater scrutiny, and processing time, for visa 
applicants from certain countries considered to be 
‘high risk’."7? It was argued that biased enforcement 
and visa statistics reinforced which nationalities 
were considered suspect.'®° Following a legal 
challenge by the Joint Council for the Welfare of 
Immigrants, the Home Office discontinued the use 
of the Streaming Tool pending a redesign of the 
process. '8" 


In Australia, s 495A of the Migration Act 1958 (Cth) 
(the Migration Act) permits the responsible Minister 
to ‘arrange for the use, under the Minister's control, 
of computer programs’ to make a decision, exercise 
a power or comply with any obligation. The Minister 
will be taken to have made a decision, exercised a 
power or complied with an obligation where the 
relevant action ‘was made, exercised, complied 
with, or done ... by the operation of a computer 
program’. 


Section 495A was inserted into the Migration Act in 
2001.18? The Explanatory Memorandum noted that 
complex decisions requiring discretionary elements, 
such as visa cancellations, would continue to 

be made by the Minister or a delegate, and that 
these types of discretionary decisions ‘do not lend 
themselves to automated assessment’.'®? The 
Parliamentary Joint Committee on Human Rights, 
however, has expressed concern that the use of 
computer programs under the Migration Act may 
lead to the arbitrary deprivation of individuals’ 
liberty. In considering proposed changes to the 
Migration Act in 2018, the Committee queried 
whether an amendment to the Migration Act would 
result ina computer program exercising ministerial 
discretion, which may have led to the unlawful and 
arbitrary deprivation of an individual's liberty.'* 


(b) The impact of Al-informed decision 
making on society 


It is increasingly recognised that there is a distinct 
risk of collective, or societal, harm from processes 
that use Al. Many stakeholders highlighted this 
point.'® The potential systemic harm ranges from 
the replication of historical societal inequality 
through algorithmic decision making, entrenching 
disadvantage,'* to the destabilisation of voting in 
democratic elections.'®” 


A number of commentators have expressed 
concern that the demonstrated capacity of Al to 
cause harm is disproportionately experienced 

by people who are already vulnerable and 
marginalised.'®® The use of machine learning and 
predictive algorithms, to mine big datasets and 
make predictions about how a person may act, 
can have serious consequences, including to deny 
employment opportunities to certain groups of 
people, and even to deprive some people of their 
liberty.'®° The systemic impact of the use of Al, 
particularly automation, in government services has 
been recognised as a distinct challenge to public 
and administrative law—laws that protect human 
rights,1%° 


Experts in computer science have repeatedly 
identified the implications of the ‘scored society’, 
where Al-powered decision-making tools ‘can 
narrow people's life opportunities in arbitrary 

and discriminatory ways’, often unseen and 
undetectable, without any form of oversight." 

The Rapporteur for the Council of Europe's Expert 
Committee on Human Rights Dimensions of 
Automated Data Processing and Different Forms of 
Artificial Intelligence, Professor Karen Yeung, warns 
of 


the collective and cumulative impacts of 
contemporary applications of data-driven 
technologies which, when undertaken 
systematically and at scale may, over time, 
seriously erode and destabilise the social 
and moral foundations that are necessary 
for flourishing democratic societies in which 
individual rights and freedoms can be 
meaningfully exercised.'%" 


The Rapporteur urged a focus on ‘our collective 
responsibility to attend to the socio-technical 
foundations of moral and democratic freedom’ 
and on the way in which the cumulative impact of 
these types of technologies could ‘fundamentally 
undermine the “moral and democratic commons” 
... without which human rights and fundamental 
freedoms cannot, in practice, be realised or 
asserted’. 


4.5 A framework for regulation 


Stakeholders clearly acknowledged that Al-informed 
decision making should comply with human rights. 
There is also strong support in the Australian 
community for regulatory reform to be pursued, 
where necessary, to achieve this aim.'™ 


The Commission recognises that there are 
important federal and state government initiatives 
related to accountable Al-informed decision 
making that are running alongside this Project. The 
recommendations in this Final Report are intended 
to complement that work. 


The UN High Commissioner for Human Rights has 
advocated a ‘smart mix’ of regulatory measures 
to protect human rights in the context of new 
and emerging technologies such as Al.1 This 
means considering how to protect human rights 
most effectively at every important stage of the 

Al ‘lifecycle’'—that is, in designing, developing and 
using Al-informed decision-making systems (see 
Figure 2, below). 
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Figure 2: Simplified lifecycle of AIS 
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In the remainder of Part B, the Commission 
considers how regulation can be most effective at 
each of the relevant stages of the Al lifecycle. 


(a) Addressing challenges to regulation 


Some stakeholders referred to challenges in 
developing effective regulation to promote 
accountability in Al-informed decision making.'°” 


Industry stakeholders in particular were wary of 
regulation that may stifle innovation or weaken 
the digital economy.'”® Some objected to any form 
of regulation in this area; others had more specific 
concerns that regulation may not be capable of 
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Design phase 
The business case is turned 
into design requirements 
for engineers 
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Training and test 
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and test model 


responding to technical challenges associated with 
ALS 


On the other hand, laws already regulate many 
forms of government and non-government decision 
making. Whether Al is used in those decision- 
making processes should not dilute or displace 
those laws. The first step should be to apply these 
existing laws more effectively. As the High Court 

of England and Wales observed in the first legal 
challenge to the use of facial recognition technology 
in 2019: 


The fact that a technology is new does not mean 
that it is outside the scope of existing regulation, 
or that it is always necessary to create a bespoke 
legal framework for it.7°° 


The Commission supports growing Australia’s digital 


economy. However, that objective does not justify 
opposing any or all regulation in this area. Laws to 


promote accountability and to protect human rights 


already exist in Australia and internationally. In 
developing and using new forms of technology, we 
must ensure that those most basic laws continue to 
be observed. 


While laws already govern many forms of decision 
making, especially by government, some reform is 
needed to ensure that human rights are upheld in 
the use of Al-informed decision making.” 


(b) Applying a human rights approach 


This Project has used the human rights framework 
to understand some of the policy and legal 
challenges posed by Al-informed decision making, 
and to frame an appropriate response. 


Stakeholders generally supported this approach. 
Some argued that a human rights approach can 
guard against the risks posed by Al-informed 
decision making, such as the ‘potential to lose 

sight of the fundamental dignity of human beings’ 
when people are treated as ‘data points rather than 
individuals’.2° 


This message is also reflected in recent opinion poll 
research. In July 2020, the Commission engaged 
Essential Media to undertake polling on trust in the 
use of Al to make decisions. Over three quarters of 
people surveyed said that they would have more 
trust in an automated government decision if 
there were stronger oversight measures, such as a 
clear right of appeal, human checks, limitations on 
personal information sharing within and outside 
government, and stronger laws to protect human 
rights.204 


When the Australian Government makes 
an automated decision that affects you: 


87% 


support a right to 
appeal the decision 


Source: National survey of 1,058 Australian adults. Essential Research, The Essential Report—Human Rights Commission (29 July 2020) 


want reasons for 
the decision 


85% 


want to know the 
decision was 
automated 


205 
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In the Office of the Australian Information 
Commissioner's Australian Community Attitudes to 
Privacy Survey 2020, survey participants strongly 
favoured accountability when Al is used in decision- 
making processes. 84% of participants strongly 
agreed that Australians have a right to know 

that a decision affecting them is made using Al 
technology; 78% believed that when Al is used to 
make, or assist in making, decisions, individuals 
should be informed what factors and personal 
information any algorithm considered and how 
these factors were weighted.” The survey also 
found that 82% of Australians consider there 
should be a right to human review of any decision 
made using Al, ‘even if this costs the organisation 
money’.2°” 


Similarly, the majority of individuals recently 
surveyed by the Monash Data Futures Institute 
indicated they ‘totally agreed’ with the need for 
industry guidelines and/or new legislation to 
address possible harms associated with the use 
of Al, including to ‘make sure Al systems are safe, 
trustworthy, and aligned with human values’.7°% 
That research also suggested the need for bodies 
that are trusted in this area to take a more 

central role in policy and regulation relevant to 
Al. One of those was the Australian Human Rights 
Commissioner, and survey respondents identified 
the Commissioner as one of the most trustworthy 
figures to manage the development and use of the 
Alin the best interests of the public.?% 


In this Final Report, the Commission primarily 
applies international and domestic human rights 
law.?'° This Report also refers to authoritative 
guidance, including the UN's Guiding Principles on 
Business and Human Rights. 


International human rights law is almost universally 
accepted as an authoritative legal standard.?"' Over 
the last 70 years, it has been remarkably adaptable, 
providing important guidance in protecting humans 
from a wide range of harms. As is increasingly 
recognised internationally,?'* ensuring international 
human rights law principles are effectively applied 
to Al-informed decision making presents a strong 
foundation for any liberal democratic government's 
response to the rise of Al. 


While there might be some gaps in the existing 
body of international human rights law which need 
to be filled, it is not apparent that an entirely new 
human rights framework needs to be developed to 
address the rise of Al.2'? The Commission cautions 
against starting a new debate about ideas such as 
‘fairness’ in the use of Al without paying due regard 
to the existing requirements of international human 
rights law.?"4 


International law requires nation states to provide 
for an effective remedy where an individual suffers 
a human rights violation. Effective remedies 

include judicial and administrative remedies, such 
as ordering compensation or an apology, and 
preventive measures that may include changes to 
law, policy and practice.2'> Effective remedies for 
human rights breaches fall under the accountability 
principle, which is central to a human rights 
approach.2'® A human rights approach also requires 
effective mechanisms to prevent human rights 
being violated.?"” 


In the following chapters, the Commission seeks 
to apply three key principles to how the Australian 
Government and private sector should design, 
develop and use Al in decision making that affects 
people’s human rights: 


* International human rights should be 
observed. The Australian Government 
should comply with human rights in its own 
use of Al, and it should also ensure that 
human rights protections are enforced for 
all entities that use Al. 


Al-informed decision making should be 
used in ways that minimise harm. Al- 
informed decision-making systems should 
be tested before they are used in ways that 
could harm individuals, and those systems 
should continue to be monitored as they 
operate. 


Al-informed decision making should 

be accountable in how it is used. An 
individual affected by an Al-informed 
decision should be given reasons for the 
decision, to enable them to understand 
the basis of the decision, and they should 
be able to challenge any decision that they 
believe to be wrong or unlawful. 


There was support among stakeholders for the 
adoption of these three principles.?'* This approach 
also draws on work in other jurisdictions that have 
sought to apply the human rights framework to 
guide regulatory responses to the use of Al, such as 
the Council of Europe.?'® 


(c) Identifying the correct regulatory object 


The human rights impact of Al depends on how it is 
used. As Al can be used in almost every conceivable 
form of decision making—banking, social security, 
criminal justice, employment, to name just a few 
areas—regulation should be framed by reference to 
the contexts in which Al is used. 


(i) Regulating technologies or outcomes? 


One option is to seek to regulate Al itself, by 
regulating particular technologies and techniques 
associated with Al. There are some examples of 
this approach in proposals overseas, such as the 
draft algorithmic accountability laws highlighted in 
Box 1.?2° 


Generally speaking, the Commission opposes 

that approach for three main reasons. First, the 
current and potential use of Al is almost limitless, 
so it would be very difficult to identify general legal 
principles that would apply to all scenarios. 


Secondly, Al is not a term of art. This makes it 
difficult, if not impossible, for legislation dealing 
with Al to be drafted with sufficient precision to 
give clarity and certainty to those who must follow, 
apply and enforce it. 


Thirdly, people are primarily affected by how 
technology is used. It can stifle innovation 
unnecessarily to require that a particular activity 
(such as making a decision) be undertaken only via 
a single means, or a single technology, or a single 
way of applying that technology. 


Instead, the focus of regulation should be on 
outcomes. Under this approach, a law might require, 
for example, that a particular type of decision be 
procedurally fair and take into account certain 
matters, but the law would not seek to constrain 


the decision maker in precisely how they must 
afford procedural fairness nor precisely when, 
where and how the particular matters must be 
considered. A further benefit of this approach is 
that it directs attention towards how current laws, 
which have long existed to protect human and 
consumer rights, could be applied more effectively. 


However, there are exceptions to this general 
approach. Some uses of Al pose such a significant 
risk to human rights that urgent, direct regulation is 
needed. 


Some stakeholders suggested that specific laws 
might be needed regarding the use of Al in high- 
risk areas such as refugee status determinations,7' 
or autonomous weapons.” Chapter 9 considers 
the use of facial recognition and other types of 
biometric surveillance technology in contexts such 
as policing.??? 


(ii) A contextual approach 


Regulation must take account of what decisions are 
made using Al. The Allens Hub for Technology, Law 
and Innovation pointed out that 


what is important is context. This includes both 
the technological context (for example, the use of 
machine learning or other data-driven inferencing 
techniques) and the broader context (such as 
whether a decision is made in the public or private 
sector). It is this context that should determine any 
legal or regulatory response.? 


In considering this context, a number of factors 
need to be taken into account. As the Al Now 
Institute observed, the starting point should be 
to consider ‘the perspective of the individuals 
and communities who are at the receiving end 
of these systems’.*#° The ‘scope of regulation 
should be determined based on the nature and 
impact of the Al system’, with a particular focus on 
decision-making systems that have ‘an impact on 
opportunities, access to resources, preservation 
of liberties, legal rights, or ongoing safety of 
individuals, groups or communities’.?”° 
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Applying this approach to the specific area of facial recognition, a proposed consumer protection law in the 
US state of Washington would apply to the use of facial recognition in decisions involving 


denial of consequential services or support, such as financial and lending services, housing, insurance, education 
enrolment, criminal justice, employment opportunities, and health care services.?2” 


In other words, regulation should be calibrated to take account of the context and especially the risk of the 
relevant area of Al-informed decision making. 


Box 2: Regulation that considers context and risk 


There is broad support in Australia and overseas for regulation targeting high-risk types of Al- 
informed decision making. 


Within the technology industry, some key companies support this approach: 


e Microsoft emphasised that a risk-based approach could consider the potential of Al to 
infringe human rights and ensure a focus on evaluating potential impacts.” 


e Amazon Web Services argued that high-risk uses of Al should be regulated differently from 
uses of Al that pose a low risk to human rights.” 


* Element Al suggested a focus on use of Al that could lead to unlawful discrimination and 
other threats to individual wellbeing. 


Several governments overseas have adopted risk-based approaches, including: 


e The European Commission's 2020 White Paper on Al proposes a risk-based approach to Al 
to ensure that any regulatory intervention is proportionate.??' ‘High-risk’ applications of Al 
include those in sectors such as health, security or justice, and where the application of the 
Al can result in discrimination.?°2 


New Zealand's Algorithm Charter includes a commitment ‘to take a particular focus on those 
algorithms that have a high risk of unintended consequences and/or have a significant 
impact if things do go wrong, particularly for vulnerable communities’. Where an algorithm 
is employed by a government agency in a way that might significantly affect the ‘wellbeing of 
people’, or there is a ‘high likelihood many people will suffer an unintended adverse impact’, 
the Charter must be followed.*¥ This requires the algorithm to be used transparently, be 
focused on people, and take into account privacy, ethics and human rights. 


The Canadian Directive on Automated Decision-Making seeks to reduce the risk of Al- 
informed decision-making systems for individuals.** It requires government departments 
to undertake an impact assessment prior to deploying an automated decision-making 
system.*° This assessment considers risk factors such as the rights, health or well-being 

of individuals or communities. Where a system is assessed as having a ‘high impact’— 
decisions that will ‘often lead to impacts that are irreversible, and are perpetual’—Al cannot 
be deployed without specific safeguards, such as ‘human intervention points during the 
decision-making process’, with the final decision made by a human.??” 
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(d) Promoting accountability 


An Al-informed decision is first and foremost a 
decision. It is certainly no less a decision than a 
conventional decision—that is, a decision where 
Al is not materially used in the decision-making 
process. 


Accountability involves ensuring that the law is 
followed in a decision-making process. It includes 
both a corrective function, facilitating a remedy 
for when someone has been wronged, as well as 
a preventive function, identifying which aspects of 
a policy or system are working and what needs 
adjustment.?38 


The Commission has identified five key questions 
that need to be answered in the affirmative to 
promote accountability in respect of Al-informed 
decision making. These do not impose additional 
requirements on Al-informed decision making, as 
compared to other decisions. Rather, considering 
these questions in the development of any new 
Al-informed decision-making system will help to 
ensure that accountability can be maintained when 
the system is used in practice. 


In the Commission's consultation process, 
stakeholders agreed on the central importance of 
Al-informed decision making being accountable, 
and on the elements of accountability set out by the 
Commission.” The remaining chapters in Part B 
seek to promote accountability in the use of Al- 
informed decision making. 


Box 3: How to promote accountability 
in Al-informed decision making? 


The following five questions will help 
determine whether an Al-informed 
decision-making system is accountable. 


1. Does the Al-informed decision- 
making system produce lawful 
decisions? Decision making must 
comply with all laws applicable to 
the decision making in question. 
For government decisions in 
particular, there must usually 
be an explicit source of legal 
authority for the decision. 


2. Is the decision making transparent? 
There should be transparency 
about when and how government 
and other bodies use Al to make 
decisions. 


3. Can reasons or an explanation 
be provided for the decisions? 
It is always good practice, and 
it is sometimes required by 
law, to produce reasons or an 
explanation for decisions. 


4. Is it clear who is legally responsible 
for a decision? While legal liability 
for decision making is usually 
clear, there are some situations 
where this needs to be made 
explicit, especially in Al-informed 
decision making. 


5. Is there appropriate human 
oversight and review? It is 
important to include human 
review to correct for errors and 
other problems in an Al-informed 
decision-making process, and for 
humans to monitor and oversee 
the use of Al at the system level. 
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5.1 Summary 


This chapter focuses on how the law can ensure that government complies with human 
rights, and acts accountably, where it uses Al to make administrative decisions. It 
addresses four key elements of accountability. 


First, there needs to be a rigorous process of planning and testing before government 
uses Al to make administrative decisions. In light of leading research on algorithmic and 
human rights impact assessments, the Commission recommends legislation to require 
government bodies to undertake a thorough process of planning, testing, monitoring 
and evaluation in respect of any proposed new Al-informed decision-making system. 


Secondly, greater transparency is needed in the use of Al by government. The 
Commission recommends that government bodies be required to notify any affected 
individual where Al is materially used in a process to make an administrative decision. 
In addition, at the systemic level, this chapter urges an independent audit of all current 
or proposed use of Al-informed decision making by the Government. 


Thirdly, individuals affected by administrative decisions generally have a right to reasons 
for those decisions. Reasons help an affected individual understand the basis of an 
administrative decision, and to challenge that decision if necessary. The Commission 
recommends a Clarification of the law to ensure that any existing right to reasons 
continues to apply to Al-informed administrative decisions. 


Fourthly, this chapter considers the role of human oversight and review in respect 

of Al-informed administrative decisions. The Commission recommends that, where 
merits review is not already available, people should be able to appeal Al-informed 
administrative decisions to an appropriate independent merits review tribunal. 
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5.2 Lawfulness 


RECOMMENDATION 2: The Australian 
Government should introduce legislation 
to require that a human rights impact 
assessment (HRIA) be undertaken before 
any department or agency uses an Al- 
informed decision-making system to 
make administrative decisions. 


An HRIA should include public 
consultation, focusing on those most 
likely to be affected. An HRIA should 
assess whether the proposed Al-informed 
decision-making system: 


(a) complies with Australia’s 
international human rights law 
obligations 


(b) will involve automating 

any discretionary element of 
administrative decisions, including 
by reference to the Commonwealth 
Ombudsman's Automated decision- 
making better practice guide and other 
expert guidance 


(c) provides for appropriate review of 
decisions by human decision makers 


(d) is authorised and governed by 
legislation. 


A government agency requires legal authority to 
make a decision. The vast majority of decisions 
made by government, especially those that affect 
the rights and interests of people, are known as 
‘administrative decisions’. Legal accountability for 
those decisions is governed by administrative law. 


Some government decisions are not ‘administrative 
decisions’ or, for other reasons, they may fall 
outside the ordinary administrative law 
accountability regime. For example, a decision 

by a government minister to introduce a bill into 
parliament or to adopt a particular policy generally 
would not be subject to administrative law 
accountability. 


The legal authority for administrative decision 
making has two elements: 


e the authority, or source of power, to make 
an administrative decision in a particular 
area 


* any requirements for how the decision may 
be made. 


Generally, the law dealing with these issues will 
apply, regardless of how the relevant administrative 
decisions are made, or the technology used to 
make the decisions. 


Given the risks associated with Al and especially 
automation, the Commission considers that 
careful planning should be undertaken before 
the Government adopts an Al-informed decision- 
making system for administrative decisions. 
Specifically, the Commission recommends that 

a human rights impact assessment (HRIA) be 
mandatory before a new Al-informed decision- 
making system is adopted by a government body 
to make administrative decisions. 


(a) Authority or source of power 


Government officials need to be granted legal 
power or authority before they can make 
administrative decisions. That source of power, 
which is usually provided for in legislation, will 
generally specify: 


e the scope of the decision-making power— 
in other words, the types of administrative 
decisions that may be made 


* who may exercise the power 


* any particular conditions or requirements 
on how the power should be exercised. 
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For example, legislation may give the Deputy 
Commissioner of Taxation power to decide whether 
to grant a taxpayer an extension of time to pay 

a tax debt. The legislation may allow the Deputy 
Commissioner of Taxation to delegate that decision- 
making power to certain office holders in the 
Australian Tax Office, and the legislation may also 
specify that a taxpayer should be given a hearing 
before a decision is made that affects them. 


It would be unusual for such legislation to specify 
the technology that may or may not be used in the 
decision-making process. However, this is starting 
to change. For example, some legislation authorises 
the use of ‘computer programs’ to make certain 
types of administrative decisions in areas such as 
tax and migration.” Some doubt, however, that 
such provisions permit the use of fully-automated 
decisions.” 


Stakeholders supported the suggestion that the 
Government should be required to legislate for the 
use of any Al-informed decision-making system and 
include adequate human rights protections.” 


It was noted that some legislation already governs 
administrative decision making, regardless of the 
technology used in the decision-making process. 
Such legislation generally could be applied to Al- 
informed decision making, with any amendments 
made as needed.”*? 


(b) Al, automation and the problem 
of discretion 


Government decision makers rarely have complete, 
unfettered discretion regarding how they make 
administrative decisions. The decision maker 
almost always will be required to follow certain 
processes in making decisions. Those processes are 
set out in a combination of legislation, the common 
law and policy documents. 


Many of the legal requirements that an 
administrative decision maker must follow, such 
as taking into account certain factors and ignoring 
other factors, may present no particular barrier to 
the use of Al in decision making. However, some 
requirements can present difficulties when Al is 
used in the decision-making process. 


Among the various forms of Al, automation 

is increasingly being used by government to 
improve the consistency, accuracy and efficiency 
of administrative decisions.2“ These are legitimate 
goals. Nevertheless, the use of Al, and especially 
automation, in government decision making 

can present challenges for human rights and 
administrative law—especially where a decision 
involves the exercise of discretion. 


The Parliamentary Joint Committee on Human 
Rights suggested that laws allowing for ‘computer 
programs’ to make administrative decisions could 
disproportionately affect human rights where 

the decision involves an element of discretion.”4° 
This has been echoed by a number of expert 
commentators and civil society bodies.” 


Similarly, the Hon Justice Melissa Perry has 
observed that discretion in administrative decision 
making can involve ‘complex and subtle questions’ 
that may be ‘beyond the capacity of an automated 
system to determine’. She warned: 


It is not difficult to envisage that the efficiencies 
which automated systems can achieve, and 

the increasing demand for such efficiencies, 
may overwhelm an appreciation of the value of 
achieving substantive justice for the individual. 
In turn this may have the consequence that 
rules-based laws and regulations are too readily 
substituted for discretions in order to facilitate 
the making of automated decisions in place of 
decisions by humans.”” 


On the other hand, the Hon Robert French AC has 
observed that Al-informed decision-making systems 
can exercise ‘a kind of discretionary function’, 

such as a machine learning system that predicts 

or determines outcomes by the application of an 
algorithm.” 


In guidance to government agencies issued in 
2019, the Commonwealth Ombudsman urged 
caution about some aspects of automation in 
administrative decision making: 


It is possible for an automated system to make 
decisions by using pre-programmed decision- 
making criteria without the use of human 
judgement at the point of decision. The authority 
for making such decisions will only be beyond 
doubt if specifically enabled by legislation.” 


The Ombudsman advised that automating a part of 
a government decision-making process will not be 
suitable where it would: 


* contravene administrative law 
requirements of ‘legality, fairness, 
rationality and transparency’ 


contravene privacy, data security or other 
legal requirements (including human rights 
obligations) 


compromise accuracy in decision making, 
or 


e ‘significantly undermine public confidence 
in government administration’.?°° 


Some overseas jurisdictions have established 
specific safeguards for the use of automation in 
decision making. For example, the GDPR prevents 
individuals, with some exceptions, from being 
subjected to a decision ‘based solely on automated 
processing, including profiling’ where that decision 
produces a legal or similarly significant effect.?°" 
The UK ICO's guidance on the GDPR states that 

to be considered ‘solely automated there must 

be no human involvement in the decision-making 
process',?> 


Other jurisdictions have established risk 
assessment frameworks to avoid the harmful 

use of automation. For example, the Canadian 
Government Directive on Automated Decision-Making 
requires a public algorithmic impact assessment to 
indicate whether the proposed use of automation 
will have a low, moderate, high or very high impact 
on individuals, including their human rights.?°3 


There are more onerous requirements for high- 
risk activities. For instance, where the decision in 
question is assessed to have a high or very high 
risk, it will be necessary to provide for ‘specific 
human intervention points during the decision- 
making process; and the final decision must be 
made by a human’.** A similar risk assessment 
process has more recently been proposed for the 
European Union.?°° 


In the course of the Commission's consultation, 
some stakeholders expressed concern about 

the use of automation in at least some forms of 
administrative decision making. This included 
situations where the risk to human rights is 
particularly high, such as in the use of autonomous 
weapons, or where the exercise of discretion is 
necessary.?°° 


(c) Benefit to the community and 
consultation 


Introducing legislation to enable Al to be used in a 
particular area of administrative decision making 
does not necessarily make it desirable to use Al, 
even where the legislation includes safeguards. 
The Commission also consulted on mechanisms to 
require the Government to assess whether, in the 
particular situation, the use of Al is likely to lead 

to better—more reliable, more efficient, safer— 
administrative decisions than existing or alternative 
ways of making those decisions. 


Some stakeholders thought a cost-benefit analysis 
could be helpful in determining whether the use 
of Al would be appropriate.” However, most 
stakeholders instead favoured a more explicit 

and targeted human rights assessment.*°° The 
Commission agrees with this view. 


There was strong support for community 
consultation in any such assessment process, with 
stakeholders noting that this could help to build 
public trust and transparency.” Several referred 
to learning from past experience, including the so- 
called ‘Robodebt' process for automated recovery 
of social security debts, as evidence of the need 
for public consultation, both with vulnerable 
consumers and with those charged with deploying 
Al-informed decision making.?© 
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Stakeholders considered that public consultation 
should take place before and during the early 
stages of implementation. In particular, there 
was support for using human rights by design 
principles,7°’ and processes such as algorithmic 
impact assessments or HRIAs.76 


HRIAs are increasingly used in the context of Al, 
including algorithmic and automated decision 
making. Some initiatives have arisen from the 
private sector; others from government2® or 
non-government organisations.” For example, the 
Australian Government's Al Ethics Principles, refer 
to Al systems respecting human rights, throughout 
the Al lifecycle, including the careful consideration 
of risk.2%& 


Asmaller number of stakeholders considered 
public consultation to be impractical or 
unnecessary, pointing to problems of cost and 
time.?6” 


(d) Conclusion: need for mandatory human 
rights impact assessments 


The Commission recommends the Government be 
required to undertake an HRIA before adopting a 
new Al-informed decision-making system to make 
administrative decisions. 


This approach aligns with the advice of expert 
authorities, such as the Council of Europe's 
Commissioner of Human Rights, who recently 
recommended a legal framework for HRIAs that are 
directed towards Al-powered systems ‘acquired, 
developed and/or deployed’ by public authorities.2@ 


While administrative decisions are generally made 
by government agencies, there is an increasing 
trend for decisions to be made by government 
business enterprises and non-government entities. 
The Commission considers that an HRIA should 

be required for the use of Al in all administrative 
decisions made by or on behalf of government. 
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The Commission recommends that an HRIA in this 
area include at least four key elements. 


First, the assessment should take a risk-based 
approach, drawing on experience elsewhere. 

For example, in one of the first challenges under 
human rights law of its kind, a Dutch court found 
the use of an automated risk assessment tool to 
prevent and detect welfare fraud breached the 
right to family and private life under the European 
Convention on Human Rights.*6° 


In evidence submitted to those court proceedings, 
the UN Special Rapporteur on extreme poverty 
and human rights noted the rapid expansion 

of digitisation of the welfare state and, in 

this particular case, observed that the tool 

was predominantly used in areas with high 
concentration of poorer and vulnerable groups.?”° 


The Special Rapporteur identified specific 
concerns with this tool, for instance the absence of 
procedural safeguards regarding its use and a lack 
of transparency, including government denial of 
access to information about the risk model, such 
as the algorithm it used, the precise indicators and 
input data the model used, and the exact output 

it produced.2”' The Special Rapporteur made the 
general conclusion that ‘digital tools are being 
mobilised to target disproportionately those groups 
that are already more vulnerable and less able to 
protect their social rights’.?”2 


The Australian Government should rigorously 
assess the risks associated with using Al, and 
especially automation, in any administrative 
decision-making process. Particular attention 
should be directed towards expert guidance, 
especially the Commonwealth Ombudsman’s 
Automated decision-making better practice guide. 


Secondly, the Government should consult the 
public on the proposed new system for making 
decisions. When it comes to Al-informed decision 
making, the importance of consultation with the 
public, particularly those who are most likely to be 
affected, has been widely recognised. 


The Australian Government's Al Ethics Principles 
contemplate consultation with stakeholders to 
ensure Al systems are ‘inclusive and accessible’ 

and do not unfairly discriminate.?”3 Similarly, the 
Council of Europe Human Rights Commissioner 
has recommended public consultation at various 
stages of the deployment of an Al system, and 

at a minimum during the procurement and HRIA 
stages.””4 The NSW Government has also noted that 
co-designed Al services will build public trust: 


Community consultation and collaboration on 
service design is essential. Communities have the 
right to influence government decision-making 
where those decisions, and the data on which they 
are based, will have an impact on them.?’5 


Thirdly, the Government should form a considered 
view about the human rights impact of the 
proposed use of Al, and especially automation, in 
the specific area of administrative decision making, 
and proceed only if there is sufficient confidence 
that the system would not breach human rights. 
This proposed assessment could draw from the 
strengths of analogous processes, such as privacy 
impact assessments under Australian privacy 
law.?’6 Similarly, modern slavery legislation requires 
human rights due diligence and risk assessment 
processes to be conducted.?’” 


Fourthly, where the Government decides to 
proceed with an Al-informed decision-making 
system for administrative decisions, the HRIA 
should ensure there is legislation that regulates 
this process, including any safeguards necessary to 
protect human rights. 


Finally, a related question is: How should an HRIA be 
undertaken? Some types of HRIA are already part 
of Australian law and practice. These are a part 

of Australian parliamentary processes;?” privacy 
impact assessments under privacy legislation;?”? 
and as part of human rights due diligence under 
modern slavery legislation. 


When an Australian Government department or 
agency undertakes an HRIA, often it would be 
practical to do so as part of the same process that is 
followed in preparing a ‘statement of compatibility’ 
under pt 3 of the Human Rights (Parliamentary 
Scrutiny) Act 2011 (Cth).2°' The proposed Al Safety 
Commissioner (see Part C) could provide guidance 
for government on this issue. 
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5. Legal accountability for government use of Al 


5.3 Transparency 


RECOMMENDATION 3B: The Australian 
Government should introduce legislation 
to require that any affected individual 

is notified where artificial intelligence 

is materially used in making an 
administrative decision. That notification 
should include information regarding how 
an affected individual can challenge the 
decision. 


RECOMMENDATION 4: The Australian 


Government should commission an audit 
of all current or proposed use of Al- 
informed decision making by or on behalf 
of Government agencies. The Al Safety 
Commissioner (see Recommendation 22), 
or another suitable expert body, should 
conduct this audit. 


The Commission uses the term ‘transparency’ to 
refer to people being made aware when Al is used 
in decision making that affects them. Transparency 
about how the government is making decisions, 
including through the use of Al, can help individuals 
protect their basic rights and is central to the 
principle of open government.?®? 


The Australian Council of Learned Academies 
(ACOLA), in its detailed report on the effective 
and ethical development of Al, highlighted the 
importance of transparency and explainability: 


To establish public confidence, it will be necessary 
to provide the public with an explanation and 
introduction to Al throughout the initial adoption 
stage.?3 


Similarly, the Australian Government's Al Ethics 
Principles include a ‘transparency and explainability’ 
principle to promote ‘responsible disclosure’ so 
that people know when they are being significantly 
affected by the use of Al.?°4 


The Commission makes two key recommendations 
to improve the transparency of government use 

of Al. First, the Government should be required to 
notify any affected individual where Al is materially 
used in a process to make an administrative 
decision. Secondly, there should be an independent 
audit of all current or proposed use of Al-informed 
decision making by the Government, to ensure the 
quality and safety of such systems. 


(a) Notification of the use of Al 


The Discussion Paper suggested a new notification 
requirement: that an affected individual be 
informed where Al is materially used in a decision 
that has a legal, or similarly significant, effect on the 
individual's rights.” There was general support for 
this proposal.7°° 


Stakeholders noted that a lack of transparency 
about the use and operation of Al-informed 
decision-making systems can make it difficult for 
individuals to enforce their rights. This restricts 

the ability of an individual to challenge a decision 
and therefore goes against fundamental values of 
administrative law. Transparency regarding the use 
of Al in decision making can also build public trust, 
and address historical breaches of that trust.?8 


Some considered that simply notifying affected 
individuals that Al has been used in a decision- 
making process may be insufficient. It was 
suggested that notification should be accompanied 
by additional information—such as how to 
challenge the decision or seek a remedy, or more 
specific detail regarding how the decision-making 
process relied on AI.?8 Notification that does little 
to inform individuals in a meaningful way, or is 
simply ‘default information’, may just cause ‘notice 
fatigue’.2°° 


Other stakeholders raised cost concerns. There 
could be a high number of notifications as Al is 
used more and more in decision making. While 
acknowledging the cost, others considered that this 
should not deter a notification requirement,2”° with 
some pointing to existing notification requirements 
in other important areas of government decision 
making, such as in some forms of healthcare.” 
Exceptions are sometimes used to avoid the risk 
that the notification requirement could pose a 
burden that is disproportionate to the benefit for 
those notified. 


(b) An audit of government use of Al 


The Commission's consultation suggested a more 
general need for better information about how the 
Australian Government uses Al in decision making. 
The Discussion Paper proposed a comprehensive 
audit of how Al is being used in government 
decision making, and the existing protections to 
ensure that such Al-informed decision making 
upholds human rights. 


There was support for this proposal.?% 

A comprehensive audit was seen as beneficial in 
promoting transparency of government use of Al, 
which in turn could build trust and enable citizen 
participation in the evolution of the government's 
technology agenda.” 


Others observed that such an audit would be 

a major undertaking and argued that the focus 
should instead be on building capacity across 
government to understand and address the risks of 
Al-informed decision making.” Some stakeholders 
argued that while an audit may be important, it is 
just as important to continue monitoring the use of 
an Al system following implementation. Information 
from ongoing monitoring should be regularly made 
public.2°> 


(c) Conclusion: steps to improve transparency 


Individuals should be made aware when they 
are the subject of Al-informed decision making, 
especially when that decision was made by 
government. 


Throughout the Project, stakeholders strongly 
favoured transparency regarding the use of Al 

in administrative decisions. It is important that 
individuals know when the use of Al is material in 
the making of decisions affecting them. 


There is evidence of emerging public expectations 
in this area. In polling undertaken for the 
Commission in 2020, 85% of survey participants 
indicated it was very important, or somewhat 
important, to be informed that a computer program 
has made an automated decision affecting them.?% 


It is also important to ease the burden on 
individuals who may seek to challenge an Al- 
informed decision affecting them. Accordingly, 
individuals should be equipped not only with 
knowledge regarding the use of Al-informed 
decision making, but also with avenues to obtain 
further information about that process, including 
whether there are any rights to challenge or appeal 
the decision that has been made. 


Transparency in this area would also be improved 
by presenting a clear picture of how the 
Government currently uses Al to make decisions. 
To this end, the Commission recommends an 
audit of all current or proposed use of Al-informed 
decision making by government. 


Australian governments are seeking to increase 

the use of Al in delivering government services, 
including as a way of advancing Australia’s 
economic recovery from the COVID-19 pandemic.?°” 
The Australian Government's Al Action Plan notes: 


Al will ... provide opportunities for governments. 
Government policy will be improved by better 

risk assessment and forecasting. Routine 
administrative tasks could be automated with 
more complex service delivery handled by 

people. This will reduce administrative costs 

for government and regulatory burden for 
businesses, while providing opportunities for more 
personalised service delivery.?% 


However, little is publicly known about how Al is 
currently used by government. There is also little 
public knowledge about how government assesses 
risks associated with Al, or the steps taken to 
address those risks. 
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More transparency has begun to emerge in other 
democracies. The New Zealand Government, for 
example, released an Algorithm Assessment Report 
in 2018, detailing the use of algorithmic decision- 
making by government departments. A similar 


exercise has recently been undertaken in the United 


Kingdom.” In other jurisdictions, civil society 
organisations have attempted to comprehensively 
map the use of Al, in circumstances where 
government disclosure has been inadequate.*"' 


The Commission recommends that the Australian 
Government appoint the Al Safety Commissioner 
(see Recommendation 22), or another suitable 
expert body, to conduct an audit of all current or 
proposed use of Al-informed decision making. 
The audit should be made public, and include 
information about what Al-powered decision- 
making tools are being used. It should also 
include information on any testing, monitoring 
and evaluation of such tools, including steps 
taken under relevant law and policy, such as the 
use of procurement rules and Privacy Impact 
Assessments. 


5.4 Explainability or a right 
to reasons 


RECOMMENDATION 5B: The Australian 
Government should not make 
administrative decisions, including 
through the use of automation or 


artificial intelligence, if the decision maker 
cannot generate reasons or a technical 
explanation for an affected person. 


RECOMMENDATION 6: The Australian 
Government should make clear that, 
where a person has a legal entitlement to 
reasons for a decision, this entitlement 
exists regardless of how the decision is 
made. To this end, relevant legislation 
including s 25D of the Acts Interpretation 
Act 1901 (Cth) should be amended to 
provide that: 


(a) for the avoidance of doubt, the 
term ‘decision’ includes decisions 
made using automation and other 
forms of artificial intelligence 


(b) where a person has a right to 
reasons the person is entitled also 
to a technical explanation of the 
decision, in a form that could be 
assessed and validated by a person 
with relevant technical expertise 


(c) the decision maker must provide 
this technical explanation to the 
person within a reasonable time 
following any valid request. 


RECOMMENDATION 7: The Australian 
Government should engage a suitable 
expert body, such as the Al Safety 
Commissioner (see Recommendation 
22), to develop guidance for government 
and non-government bodies on how to 
generate reasons, including a technical 
explanation, for Al-informed decisions. 
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Australian law generally, but not always, requires 
administrative decision makers to provide reasons 
for their decisions. 


However, it can be difficult to generate reasons 

or an explanation for the outputs—including 
inferences, recommendations and decisions— 
produced by Al-informed decision-making systems. 
Indeed, the use of Al can obscure the rationale or 
reasons for a decision, which in turn can frustrate 
a legal right to reasons.? This problem is often 
referred to as ‘opaque’ or ‘black box’ Al. 


The concept of ‘explainability’ has arisen as a 
solution to this problem. The UK ICO explores some 
of the different ways of explaining Al decisions, 

and important contextual factors that must be 
considered.?° 


There is growing awareness of the importance 

of explainable Al in underpinning accountability. 
The OECD Guiding Principles on Al, for example, 
include commitments to ‘transparency and 
responsible disclosure regarding Al systems’ and 
to guaranteeing the provision of ‘meaningful 
information, appropriate to the context, and 
consistent with the state of art’. This information 
should enable an affected person to ‘understand 
the outcome’ and to challenge that outcome ‘based 
on plain and easy-to-understand information on the 
factors, and the logic that served as the basis for 
the prediction, recommendation or decision’.2“ 


The Australian Government's Al Ethics Principles 
includes a requirement of explainability, which is 
intended to provide ‘reasonable justifications for 

Al systems outcomes’, including ‘information that 
helps people understand outcomes, like key factors 
used in decision making’.>% 


Similarly, the IEEE has acknowledged the need 
‘to characterise what ... algorithms or systems 
are going to do via transparent and traceable 

standards’, which may include: 


preferential adoption of effective design 
methodologies for building ‘explainable Al’ (XAI) 
systems that can provide justifying reasons or 
other reliable ‘explanatory’ data illuminating the 
cognitive processes leading to, and/or salient 
bases for, their conclusions.2%° 


In the Commission’s view, Al-informed 
administrative decisions should be treated like 
other administrative decisions in terms of a 
person's right to reasons. While there may be a case 
for requiring greater transparency for Al-informed 
decision making by government, at the very least 
where the right to reasons exists in Australian law, 
it should not be diluted or undermined simply 
because Al was used in the process of making an 
administrative decision. 


This section contains three recommendations to 
apply that principle: 


1. The Government should not be permitted 
to use black box or opaque Al in 
administrative decision making, if the 
result is that the relevant authority cannot 
generate reasons for their decisions. 


2. The Government should introduce 
legislation to clarify that any existing 
entitlement to reasons for a decision should 
apply to Al-informed decisions, and that 
a technical explanation for such decisions 
should be available on request. 


3. The Government should provide guidance 
on how to generate reasons for Al-informed 
decisions. 


(a) Applying the existing right to reasons 


For some administrative decisions, a right to 
reasons is provided by legislation.*°” For example, 
the Administrative Appeals Tribunal Act 1975 (Cth) 
(AAT Act) provides that a person may obtain 
reasons for most decisions to which the Act 
applies.” The Administrative Decisions (Judicial 
Review) Act 1997 (Cth) provides that reasons may be 
obtained for a broad range of other Commonwealth 
administrative decisions.°° 


In addition, freedom of information laws provide 
a right to obtain access to information that may 
help to understand the basis of an administrative 
decision, subject to exemptions.’ Regulators 
and courts can also require the production of 
information or specific documents in relation to 
complaints and disputes within their respective 
jurisdictions." 
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Some laws apply the right to reasons specifically to 
Al-informed decision making. The United States Fair 
Credit Reporting Act (1970), for example, attaches 
specific duties to consumer reporting agencies, 
which require notification of ‘adverse action’ to 

a person regarding why they have been denied 
credit. This, in turn, requires an explanation of why 
that conclusion has been reached, including where 
an automated algorithmic process has been relied 
upon to make that decision.?" 


While the GDPR has been said to include, or 
perhaps support, a right to an explanation,?" 
recent analysis from the European Parliament 
suggests the GDPR does not conclusively provide 
for an explanation for Al-informed decisions, and 
recommended further work be undertaken in this 
regard.3"4 


Stakeholders considered explainability, or the 
right to reasons, to be critical in ensuring that Al- 
informed decision making is accountable.” Where 
existing law already includes a right to reasons for 
an administrative decision, stakeholders generally 
supported the application of that right to the more 
specific context of Al-informed decision making. 


Some stakeholders urged further reform. Some 
supported a legislated right to an explanation for 
all Al-informed decisions;3"° others stated that the 
right to an explanation should not be founded in 
legislation.2"” 


Stakeholders suggested that any explanation 
should be tailored to the context of the decision, 
based on an understanding of why the explanation 
is being provided, and sufficient detail to enable 
the decision to be challenged.?"® It was suggested 
that government agencies should be guided in 
how to give reasons for Al-informed decisions.?"? 
It was also suggested that some exemptions to 
the requirement to provide reasons might be 
appropriate, such as where the balance of public 
interest militated against providing reasons, or in 
respect of certain types of Al-informed decision.*”° 


(b) Legal restrictions on opaque decisions? 


Where a decision is described as black box or 
opaque, a person cannot determine the reasons 
or basis for the decision. Given the problems 
associated with such decision making, and the 
potential for Al-informed decision-making systems 
to produce decisions without reasons or an 
explanation, the Discussion Paper put forward 
options for how Australian law might restrict such 
decision making. 


The Commission proposed a prohibition on Al- 
informed decision making that cannot produce 
‘reasonable explanations for its decisions’.?*' This 
proposal elicited both strong support, and 
opposition.?2? 


Several stakeholders sought clarification regarding 
the ‘reasonable’ explanation requirement and 
sought more guidance on when the requirement 
might apply, noting that what would be considered 
reasonable would vary between contexts and 
applications.” 


Stakeholders also expressed serious concern that 
imposing a reasonable explanation requirement 
could inhibit innovation.’ Others pointed to the 
economic and social cost of requiring reasons to be 
provided.? Finally, a small number of stakeholders 
recommended the Commission consider 
exemptions to the proposal.*?” 


Question B in the Discussion Paper also asked 
whether Australian law should impose a rebuttable 
presumption that an Al-informed decision is 
unlawful if the person responsible for the decision 
does not provide a reasonable explanation. 


Some stakeholders supported such a rebuttable 
presumption as an important human rights 
safeguard in the context of Al-informed decision 
making.” 


Others were opposed, or offered qualified support, 
subject to amendments and further clarification.?? 
Concerns included that imposing such a 
presumption could violate the rule of technological 
neutrality,” and others considered it to be a 
disproportionate regulatory imposition that would 
inhibit innovation.” 


(c) Content of reasons or an explanation 


Where a right to reasons exists, s 25D of the Acts 
Interpretation Act 1901 (Cth) (Acts Interpretation Act) 
provides further requirements about what must be 
included in those reasons. Section 25D states: 


Where an Act requires a tribunal, body or person 
making a decision to give written reasons for 

the decision, whether the expression “reasons”, 
“grounds” or any other expression is used, the 
instrument giving the reasons shall also set out 
the findings on material questions of fact and refer 
to the evidence or other material on which those 
findings were based. 


It is important to consider whether this provision 
is adequate to deal with the rise of Al-informed 
administrative decisions. 


(i) Non-technical reasons or explanation 


Generally, reasons or an explanation, such as that 
provided for in s 25D of the Acts Interpretation Act, 
are sufficient to promote accountability. 


Stakeholders emphasised the importance of an 
affected individual knowing why a decision was 
made (ie, being given reasons or an explanation) 
in ensuring accountability and human rights.?°? 
This accords with international jurisprudence and 
commentary. 


The UN Committee on the Elimination of Racial 
Discrimination, for example, recently drew the link 
between black box or opaque Al and an inability to 
challenge unlawful decision making: 


Given the opacity of algorithmic analytics and 
decision-making, in particular when employing 
artificial intelligence methods, discriminatory 
outcomes of algorithmic profiling can often be less 
obvious and more difficult to detect than those 

of human decisions and thus more difficult to 
contest.?* 


Stakeholders submitted that affected individuals 
need enough information about the decision to 
enable them to exercise a right of review.’ Some 
stakeholders warned that simply providing the 
technical basis for an Al-informed decision may do 


little to assist individuals to understand or challenge 
it.33¢ 


Several stakeholders also noted that the context 

of the decision would dictate what—and how 
much—information would be necessary to provide 
in reasons.” Some stakeholders were also wary of 
the burden that simply providing a non-technical 
explanation of an Al-informed decision would place 
on individuals, particularly if there are no additional 
structural changes to governance, such as enforcing 
human-centric design of Al-informed decision- 
making systems.?8 


(ii) A requirement for a technical explanation? 


At least in some situations, a simple lay-person’s 
explanation or reasons may be inadequate for 
Al-informed decision making. To this end, the 
European Commission's Independent High-Level 
Expert Group refers to the importance of having 


the ability to explain both the technical processes 
of an Al system and the related human decisions 
.... Technical explainability requires that 

the decisions made by an Al system can be 
understood and traced by human beings.” 


Some stakeholders supported reform to provide 
better access to technical information about an 
Al-informed decision-making system.>“° It was 
suggested that the following elements could be 
included within a technical explanation: 


e the training and validity of the data set, and 
transparency regarding the data selection, 
and any methods used to test or screen the 
data set for biases?“ 


system level information, including 
reference to collective outcomes (rather 
than focusing only on the individual 
outcome), as well as reasons, or the 
business logic, for choosing the particular 
Al decision-making tool?4 


detail regarding how the Al is being used 
by the decision maker, and detail of the 
algorithmic model or software being 
deployed.##? 


Some existing laws that support a right to reasons 
may require reform to ensure that all relevant 
technical information related to an Al-informed 
decision is supplied or able to be obtained.“ 
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Other stakeholders opposed reform, or supported 
only limited reform, regarding access to a technical 
explanation of Al-informed decisions.**° It was 
noted that ‘technical information’ could potentially 
include proprietary, confidential or personal 
information.” This could be complicated also 
because what will be relevant information will differ 
from one Al-powered system to another.” 


Alternatives to a legal obligation for a technical 
explanation were suggested. For example, 
disclosure statements setting out non-technical 
information or an overview of the decision-making 
tool could be provided, neither of which need to 
disclose sensitive details of its operation.” An 
independent professional review of an Al-informed 
decision-making system could be provided, to avoid 
widespread disclosure of the relevant technical 
information. 


(d) Conclusion: the need for administrative 
decisions to be explainable 


A failure to provide reasons for an administrative 
decision can make it difficult, or even impossible, 
to know whether the decision was fair, reasonable, 
accurate or even lawful. This can result in an 
unaccountable exercise of government power, and 
can be an affront to the rule of law.*°° 


By contrast, reasons for an administrative decision: 


* allow the affected individual to understand 
why the decision was made, putting them 
in a position to decide whether the decision 
involved an unwarranted finding of fact, or 
an error of law, that is worth challenging”! 


assist a court in determining whether a 
reviewable error of law has occurred?52 


promote real consideration of the issues, 
increasing ‘public confidence in, and the 
legitimacy of, the administrative process’. 


Where the use of Al causes an administrative 
decision to be more opaque, by obscuring the 
reasons for which the decision was made, this is a 
barrier to achieving these desirable outcomes. The 
Commission has concluded that where a right to 
reasons exists in Australian law, it should not be 
diluted or undermined simply because Al was used 


in the process of making an administrative decision. 


Numerous polls have shown the public expect Al 
to be used accountably.?** Reasons for Al-informed 
decisions can thus help build public trust in 
government use of Al. 


(i) Reasons for Al-informed administrative 
decisions 


There is a statutory right to reasons for some forms 
of administrative decision, including under s 28 

of the AAT Act. In some circumstances, policy and 
other official government advice urges government 
agencies to provide a statement of reasons even 

if they are not legally required to do so. The 
Commonwealth Ombudsman can also recommend 
a government agency remedy its omission of 
reasons, or give better reasons, in relation to action 
it has taken.3°5 


The Commission acknowledges that it is difficult or 
even impossible to generate reasons to explain the 
operation of at least some forms of Al. As ACOLA 
observed, for Al that engages in unsupervised 
learning, ‘it is, in principle, impossible to assess 
outputs for accuracy or reliability’.3°° 


However, that technical difficulty does not 

justify a failure to comply with basic principles of 
government accountability. The benefits of such 
opaque forms of Al-informed decision making 
should never be assumed to outweigh the risks of 
harm. Governments should be able to explain how 
they arrive at decisions. Therefore, the Commission 
opposes the use by government of Al-informed 
decision-making systems that cannot generate 
reasons, or a technical explanation, for the 
resultant decisions. 


The need for action is pressing—something 
illustrated by the Government's automated 
debt recovery scheme, which came to be 

known as ‘Robodebt’. In reviewing this scheme, 
the Commonwealth Ombudsman urged the 
Department of Human Services to ‘improve the 
clarity’ of the letters sent to individuals notifying 
them of a claimed debt, and to give such people 
‘better information so they understand the 
information and can properly respond to it’.?°” 
In a national survey in 2020, 88% of respondents 
considered it very important or somewhat 
important to be given reasons or an explanation 


for an automated decision made about them by the 
Australian Government.?°8 


The Commission has concluded that reform is 
needed to make clear that the use of Alin an 
administrative decision-making process does not 
negate any entitlement to reasons for that decision. 
This could be achieved by amending the Acts 
Interpretation Act to provide, for the avoidance of 
doubt, that a legal entitlement to reasons applies 
regardless of how a decision is made, including 
where automation or Al is used in the decision- 
making process. 


A similar amendment could be made also to 
existing legislation that creates a right to reasons, 
such as s 28 of the AAT Act. Ultimately, in 
considering reform in this area, government should 
focus on the nature of the decision and its impact 
on individual rights, rather than the technology 
used to make the decision. 


(ii) Content of reasons 


Section 25D of the Acts Interpretation Act sets out 
minimum requirements for the content of reasons, 
where a legal requirement to provide reasons 
already exists. In essence, this is a requirement for 
a lay explanation of the decision. 


In most situations, this would satisfy the needs 

of a person affected by the decision. However, 
sometimes it will be necessary to verify whether the 
reasons provided accurately reflect the real reasons 
a decision was made. 


The Commission recommends that, where 
requested, the relevant government agency should 
be required to produce a technical explanation 

of an Al-informed decision, in a form that can be 
assessed and validated by a person with relevant 
technical expertise. 


This reform should be implemented by amending 
s 25D of the Acts Interpretation Act. Consistent 
amendment of other relevant legislation that 
provides legal entitlements to reasons for an 
administrative decision, including s 28 of the AAT 
Act, also may be desirable. 


Many of the techniques used to explain why Al 
systems produce particular outputs are not readily 
understood by people without relevant technical 
expertise. To enable lay people—as well as 
regulators and adjudicative bodies such as courts— 
to understand why some decisions are made, it will 
be necessary for experts to assist in interpreting 
the outputs of some Al tools. In practice, it has been 
suggested that the government could make neutral 
experts available to assist affected individuals in 
understanding such technical information.?°° 


A good example of such a technical explanation was 
ordered by the Federal Court in a recent case. In 
order to determine a narrow question of consumer 
law, the primary judge relied on the expert evidence 
of four computer scientists. The experts gave 
evidence on the content and computation of data to 
explain how an online vendor's algorithm provided 
recommendations to consumers.3© 


There are many different approaches to providing 
technical explanations of Al-informed decisions.*©' 
Current research focuses on model explanation 
(explanation using a transparent and fully 
interpretable model capturing the logic of the 
opaque system); model inspection (explanation 
to understand specific properties of an opaque 
model or its predictions); or outcome explanation 
(explaining the outcome of an opaque model in a 
specific instance).3 


An explanation may consist of a number of 
different factors, such as the original datatset; 

how the system was trained on that data set; any 
risk mitigation strategies adopted; the factors, 

or combination of factors, used to determine 

an outcome or prediction; any evaluation or 
monitoring of individual or system outcomes; and 
any testing, or post-deployment evaluation, carried 
out in relation to the model.?% 


While the Commission does not prescribe what a 
technical explanation should consist of, guidance 
from an independent expert body, such as the 
Al Safety Commissioner recommended in Part C, 
would be helpful in this area. 
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5.5 Human oversight, intervention 
and review 


RECOMMENDATION 8: The Australian 
Government should introduce legislation 
to create or ensure a right to merits 


review, generally before an independent 
tribunal such as the Administrative 
Appeals Tribunal, for any Al-informed 
administrative decision. 


While Al, including automation, can bring genuine 
benefits in administrative decision making, it 

can also undermine accountability. Australian 

law generally requires an affected individual to 

be able to review administrative decisions that 
affect them.?* Almost nine out of ten respondents 
surveyed in a national 2020 poll considered it 
important to be able to appeal a government 
decision that they do not like or think might be 
unlawful.?% 


There are generally three main forms of review 
open to an affected individual who wishes to 
challenge an administrative decision: 


* internal review—where the body (usually a 
government agency) responsible for making 
the administrative decision undertakes its 
own review. Generally, the review is carried 
out by another decision maker, who is 
at least as senior as the original decision 
maker. Internal review can be made 
available either by policy or law. 


* external merits review—where an external 
body, usually an independent tribunal, 
reviews the original administrative decision. 
Typically, in external merits review, the 
tribunal ‘stands in the shoes’ of the original 
decision maker and can remake that 
decision in any way they see fit, exercising 
all of the relevant powers, duties and 
discretions vested in the original decision 
maker. External merits review is common, 
but not universal, and is available only 
where it is expressly provided for in law. 


judicial review—where a court considers 
the lawfulness of the process by which 

the administrative decision was made. If 
that process is found to be unlawful, the 
court generally will order that the decision 
be remade in accordance with the law. 
Section 75(v) of the Australian Constitution 
provides that decisions of Commonwealth 
officers must always be subject to judicial 
review, and this constitutional protection is 
supplemented by several pieces of primary 
legislation. 


In addition, many forms of administrative decision 
can be subject to oversight by bodies such as the 
Commonwealth Ombudsman and the Commission 
itself. While those bodies generally cannot require a 
decision to be changed, in many situations they can 
recommend changes to the administrative decision 
in question. 


There can be difficulties posed by Al-informed 
administrative decisions in each of these forms of 
review and oversight. This section considers those 
challenges, and recommends reform to ensure 
that Al-informed administrative decisions can be 
reviewed by an independent merits review tribunal. 


(a) Internal review and the ‘human-in-the- 
loop’ 


It is relatively rare for an administrative decision- 
making process to be wholly automated, with no 
human involvement at all. Instead, human oversight 
and intervention in such a process—commonly 
referred to as a ‘human-in-the-loop’—can identify 
and address problems that may arise through 
automation. The importance of human oversight 
and intervention is discussed more generally in 
Chapter 7. 


Some other jurisdictions provide expressly for 
internal human review of automated decisions. 

In the UK, where a significant decision is made 
solely via automation, an affected individual can 
request the data controller to reconsider the 
decision, or make a new decision that is not based 
solely on automated processing. These laws 

are understood to apply to many administrative 
decisions, as well as decisions made by private 
sector data controllers, such as banks or insurance 
companies. 


Difficult questions can arise in assigning the 
appropriate role of a human-in-the-loop. How do 
we ensure that any human intervention in the 
decision-making process leads to objectively better, 
more accurate decisions? How do we combat the 
risk that the human-in-the-loop gives no more than 
the illusion of human involvement in a decision- 
making process that is, in reality, automated? For 
instance, it has been observed that if the particular 
human-in-the-loop lacks the relevant technical and 
other knowledge to understand the decision, they 
will be ill-equipped to remedy any problems with 
the original decision.*°” 


Throughout the Commission’s consultation, 
stakeholders have emphasised the importance 

of the role played by humans in overseeing, 
monitoring and intervening in Al-informed decision 
making.” While there were differing views about 
the most effective role for humans in Al-informed 
decision-making processes, there was a higher 
level of agreement that human intervention can 
occur most usefully through the creation of review 
and appeal rights, to a human, for Al-informed 
decisions.3© 
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(b) External merits review 


The Discussion Paper asked whether reform was 
needed regarding human review of Al-informed 
decision making. Stakeholders expressed a range of 
views on this question. 


Some stakeholders specifically supported a clear 
right to merits review of administrative decisions, 
including where the decision making is wholly 
automated,?” and particularly in respect of 
decisions that may affect an individual’s human 
rights.3”! It was also observed that the review 
process should enable lessons to be learnt to 
improve the Al-informed decision-making system 
under review.?”2 


Other stakeholders opposed reform in this area. In 
particular, it was observed that, in some situations, 
human intervention can increase, rather than 
decrease, the risk of error.?” 


(c) Judicial review 


The Australian Constitution contains a minimum 
guarantee of judicial review for administrative 
decision making. This is supplemented by 
legislation and the common law. 


The Administrative Decisions (Judicial Review) Act 1977 
(Cth) (ADJR Act) is the primary statutory source of 
judicial review for Australian Government decisions. 
The ADJR Act sets out the grounds on which 

judicial review may be sought, including that an 
administrative decision was in breach of the rules 
of natural justice or procedural fairness, or that the 
decision maker failed to take account of relevant 
considerations that they were bound to take into 
account or took irrelevant considerations into 
account.?”4 Where a decision is unlawful, a court 
exercising judicial review can provide remedies, 
including by quashing or setting aside a decision or 
requiring a new decision to be made.?” 


In addition, judicial review schemes are provided for 
in other statutes, and regulate review of decisions 
made under specific legislation—for example, in the 
areas of migration and taxation. 


Al-informed administrative decision making raises 
some complex issues in relation to the availability of 
judicial review. Some wholly opaque administrative 
decisions might even be unconstitutional if the 
manner by which these decisions are made 
frustrates a court, established under Chapter III 

of the Australian Constitution, from exercising its 
judicial review function.?”6 


Another difficulty relates to when automated 
processes will result in a reviewable administrative 
decision being made. This question was considered 
in 2018 by the Full Court of the Federal Court of 
Australia in the Pintarich case, which related to a 
disputed tax debt. The taxpayer received extensive 
material from the Australian Tax Office (ATO), 
including a letter that a human decision maker at 
the ATO generated using a computer program. 

The letter included information that was less 
favourable to the taxpayer than some other ATO 
communications, regarding the taxpayer's tax debt. 
The Court decided, by majority, that the letter was 
not a reviewable ‘decision’ under the ADJR Act.?”” 


The majority judges found that in order for there 
to be a decision, ‘there needs to be both a mental 
process of reaching a conclusion and an objective 
manifestation of that conclusion’? In dissent, Kerr 
J found that applying this computation process still 
involved a reviewable decision: 


What was once inconceivable, that a complex 
decision might be made without any requirement 
of human mental processes is, for better or worse, 
rapidly becoming unexceptional. ... The legal 
conception of what constitutes a decision cannot 
be static; it must comprehend that technology has 
altered how decisions are in fact made and that 
aspects of, or the entirety of, decision making, can 
occur independently of human mental input.?” 


The broader significance and implications of 
Pintarich as a precedent are yet to emerge. On one 
view, the Court simply reinforced an important 
principle: that legally valid decisions can be made 
only by people holding actual legal authority to 
make them. 


On another view, some unfairness could arise from 
the general scenario raised by this case. That is, if 
a government agency communicates with a person 
using an automated process, and the person 

takes action in reliance on that communication, 
but the agency subsequently makes a binding 
decision that is at odds with its earlier automated 
communication, should the person not be able to 
rely on the original communication—regardless 

of whether, as a technical question of law, the 
automated communication was an administrative 
decision? 


The Full Federal Court's decision in Pintarich does 
not purport to resolve such larger questions. 
However, this case suggests that the use of Al in 
administrative decision making can affect its legal 
status, altering how, and even whether, certain 
government decision making can be reviewed. 


(d) Conclusion: a right to independent 
merits review 


Australian law generally requires an affected 
individual to be able to review administrative 
decisions that affect them.*°° Almost nine out 
of ten respondents surveyed in a national 2020 
poll considered it very important or somewhat 
important to be able to appeal a government 
decision that they do not agree with or think 
might be unlawful.” 


The GDPR contemplates internal and external 
review for automated decision making, providing 
for rights of review and natural justice. Article 22 
recognises the right of an individual ‘not to be 
subject to a decision based solely on automated 
processing, including profiling’, and provides for the 
individual to obtain ‘human intervention’ whereby 
the individual can express their view and contest 
the decision. 


Similarly, in Canada, human review must be 
available for certain forms of automated decision 
making by government, where the decisions are 
deemed to be ‘high risk’ by reference to the impact 
of the decision on factors including the rights, 
health, and economic interests of individuals or 
communities.’ 


In the Commission’s view, reform in Australia 
is needed to ensure similar rights of review are 
available for people affected by Al-informed 
administrative decisions. 


(i) Internal review and the human-in-the- 
loop 


It is good practice to include a human in an Al- 
informed decision-making process that deals with 
legal and other similarly significant rights. One or 
more humans—with appropriate expertise and 
experience—can be well placed to: 


e identify errors in the operation of the 
system 


e exercise discretion 


* assess the overall fairness of a decision 
before it is finalised, and consider whether 
it complies with human rights. 


Difficult questions can arise in ensuring that a 
human-in-the loop is well placed to improve the 
quality of an otherwise automated decision-making 
process. However, there is a growing body of 
research that provides useful advice on this issue. 
Much of this research focuses on designing systems 
that incorporate user experience from the earliest 
stages. For example, ‘human-centred’ design 
promotes a ‘deep understanding of the people, 
contexts and environments’ connected with Al- 
informed decision-making systems, to ensure they 
are designed in a way that enables people to assess 
the system's trustworthiness.?® 


The most appropriate approach is usually to 
integrate the best, most reliable features of Al 

and human decision making. Recent research 
undertaken by the European Parliament regarding 
the impact of the GDPR concluded: 


In many cases, the best solution consists in 
integrating human and automated judgements, 

by enabling the affected individuals to request a 
human review of an automated decision as well 

as by favouring transparency and developing 
methods and technologies that enable human 
experts to analyse and review automated decision- 
making.” 
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Consistently with Recommendation 2, the 
Commission considers that where the Australian 
Government is contemplating the creation of an Al- 
informed decision-making system, it should provide 
rights to internal review by a human. 


(ii) External merits review 


A critical safeguard for administrative decision 
making generally is the ability for people affected 
by such decisions to access external merits review. 
External merits review is especially important in 
respect of Al-informed administrative decisions. 


The Commonwealth Ombudsman has concluded 
that people ‘must be provided with an opportunity 
to dispute an administrative decision made by or 
with the assistance of an automated system’. The 
Ombudsman advised: 


Many administrative decisions are subject to a 
legislative review process. In other cases, the 
agency should offer an option for internal review 
by a staff member as a part of a commitment to 
fair and reasonable decision-making. External 
avenues of review should also be provided to 
customers such as the option to make a complaint 
to the Ombudsman or taking a matter to a tribunal 
or court.3°5 


External merits review is typically performed in 
Australia by independent tribunals, such as the 
Administrative Appeals Tribunal at the federal level. 
Tribunals review administrative decisions, standing 
in the shoes of the original decision maker.” 


Many forms of administrative decision, especially 
those that affect individuals’ legal or similar rights, 
are already subject to external merits review. The 
Commission concludes that external merits review 
before an independent tribunal generally should be 
available for Al-informed administrative decisions. 
This would change the existing law only in respect 
of administrative decisions that are made using Al, 
and not already subject to external merits review by 
a tribunal. 
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As with administrative decision making more 
broadly, the Commission acknowledges that there 
may be exceptions where external merits review 
by a tribunal is inappropriate. Such exceptions, 
when they apply to automated administrative 
decisions, should be rare and only where there 

is a demonstrated need for automation in the 
decision-making process and an overwhelming 
public interest against external merits review (such 
as national security). Where such exceptions exist, 
the Government should consider providing for 
alternative forms of merits review, such as via the 
Ombudsman. 


(iii) Judicial review 


Judicial review of administrative decision making is 
fundamental to any liberal democracy that upholds 
the rule of law. In Australia, it is protected by our 
Constitution. 


Generally, the use of automation and other 
forms of Al does not threaten the right to judicial 
review. However, as discussed earlier in this 
chapter, concerns have been raised about the 
implications for access to judicial review of 
‘opaque’ administrative decisions and automated 
government processes, such as computer- 
generated communications. In particular, the Full 
Federal Court's decision in Pintarich should put us 
on notice that reform in this area may be necessary 
in future, to protect access to judicial review. 


6. Legal accountability for private 
sector use of Al 


6.1 Summary 


Chapter 5 considered accountability for Al-informed decision making by government. 
This chapter addresses the need for accountability in Al-informed decisions made by 
corporations and other non-government entities. 


Everyone has a responsibility to respect human rights. Countries, such as Australia, must 
ensure there is a regulatory framework that effectively upholds human rights,?°” with 
individuals able to access a remedy if their rights have been infringed.” This can include 
administrative and judicial mechanisms to address human rights violations.?°° 


This chapter recommends several ways to promote greater accountability in Al-informed 
decision making by the private sector. In particular, people affected by such decisions 
should be: 


* made aware of the decision, and the use of Al in the decision-making process 


e provided with reasons or an explanation for the decision 


* able to appeal to a body that can review the decision and correct it, if necessary. 
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6.2 Lawfulness 


RECOMMENDATION 9: The Australian 
Government's Al Ethics Principles should 
be used to encourage corporations 

and other non-government bodies 

to undertake a human rights impact 
assessment before using an Al- 


informed decision-making system. The 
Government should engage the Al Safety 
Commissioner (Recommendation 22) to 
issue guidance for the private sector on 
how to undertake human rights impact 
assessments. 


In any system of legal accountability, the first 
question to ask is: What are the legal rules that must 
be followed? 


While some of the obligations that apply to 
administrative decisions made by government differ 
from decisions made by non-government entities, 
there is acommon core of legal requirements that 
apply to almost all decision making. For example, 

as explored in Chapter 8, it is unlawful for anyone 
to make a decision that discriminates on the basis 
of an individual's race, sex, age or other protected 
attribute. There are very few exceptions to this rule. 
Beyond this common core, the legal rules that apply 
to decision making by non-government entities 

are generally less onerous or prescriptive than for 
government decision making. 


The Commission urges that the private sector 
exercise caution in its use of Al-informed decision 
making, to avoid acting unlawfully or otherwise 
causing harm to individuals. This section of the 
chapter recommends practical measures for 

the Government to support the private sector 

to undertake human rights impact assessments 
(HRIAs), prior to using Al-informed decision-making 
systems. 


(a) What legal rules apply to non- 
government decision making? 


In addition to the common core of legal obligations 
that apply to all forms of decision making, there 
are two significant ways in which the law can 
impose obligations on the decision making of non- 
government entities. 


First, a non-government entity, which is exercising 
a power or duty of the state, will often be required 
to comply with certain accountability requirements. 
For example: 


* Government business enterprises are 
creatures of government, but have 
corporate structures and usually fall 
outside the general administrative law 
accountability regime. Their decision 
making tends to be regulated by dedicated 
legislation, supplemented by laws that 
apply to all corporations. For example, 
the powers, duties and functions of 
Australia Post are set out in the Australian 
Postal Corporation Act 1989 (Cth), with 
that legislation also setting out specific 
requirements for how certain decisions are 
made, as well as additional accountability 
requirements including an obligation to 
report to Parliament. 


Some non-government entities enter into 
contracts to carry out functions on behalf 
of government, including in sensitive areas 
such as aged care, hospitals, prisons and 
other detention facilities. It is common for 
an alternative accountability regime to be 
maintained through the contract between 
the government and the service provider, 
as well as any applicable legislation. 


While Australia does not have a federal 
human rights act or constitutional bill 

of rights, the respective human rights 
statutes in the Australian Capital Territory, 
Victoria and Queensland each provides 
that, where a non-government body is 
performing a public function on behalf of 
the government, the non-government body 
must comply with relevant human rights.?%" 
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Secondly, the law sometimes contains obligations 
for particular areas of non-government decision 
making. By way of illustration, the Corporations Act 
2001 (Cth) requires company directors and other 
senior executives to exercise their powers and 
duties, including in the way they make decisions, 
with care and diligence.*? For example, in making 
business judgments, company directors must: 


* make the judgment in good faith and for 
a proper purpose 


* not have a material personal interest in the 
subject matter of the judgment 


* inform themselves about the subject 
matter of the judgment to the extent they 
reasonably believe to be appropriate 


* rationally believe that the judgment is in the 
best interests of the corporation.?°? 


The use of Al can make it more difficult to 
determine whether corporations have complied 
with their legal obligations. A recent comparative 
study found that corporate law in Australia and 
the United Kingdom does not ‘appropriately 
cater for the use of Al by directors’, and it has 
been suggested that ‘whether the use of Al will 
exacerbate liability issues ultimately depends 
on the fashion in which courts will interpret 

the wording of statutory provisions relating to 
directors’ duties’.3* 


(b) Conclusion: importance of human rights 
impact assessments 


As a general principle, the use of Al in decision 
making should not change the legal rules that must 
be followed. 


There are two important qualifications to this 
general principle. First, where the use of Al 
materially changes the character of the decision- 
making process, it may be necessary to apply new 
or additional rules to that process. For instance, 
conventional decision making may enable the 
consideration of a limited range of factors. 
However, Al can sometimes enable new factors to 
be considered, through techniques such as big data 
analysis. In this situation, law and policy makers 
may need to consider whether to permit those new 
factors to be considered. 


Secondly, the use of Al can sometimes affect— 
positively or negatively—the reliability of a decision- 
making process. For example, in some areas of 
precision medicine, Al-powered diagnostic devices 
can be more accurate than conventional methods. 
In other areas of decision making, such as with 
some types of facial recognition, the use of Al 

can result in decisions that are less accurate than 
alternative methods. In either situation, the 
relative accuracy of using Al may necessitate new 
legal rules regarding its use in those specific areas. 


Outside of those scenarios, non-government 
bodies should exercise caution in their use of Al 

in decision making that affects people's legal or 
similarly significant rights. The Commission strongly 
encourages the private sector to follow human 
rights by design principles, as outlined in Chapter 

7, whenever considering, developing or using an Al- 
informed decision-making system. 


In particular, companies should be encouraged 

to undertake human rights impact assessments. 

As explored in greater detail in Chapter 7, these 
assessments can help businesses and other non- 
government entities identify and address human 
rights and related risks associated with the use of Al 
in decision making. 


The Commission recommends that the Australian 
Government encourage the private sector to 
undertake HRIAs. One way of doing this would be 
through the Government's Al Ethics Framework, 
which includes voluntary Al Ethics Principles 

that ‘can inform the design, development 

and implementation of Al systems’.°° As the 
Government has noted, this framework is still being 
further developed, and the Government intends to 
flesh out some of its high-level guidance through 
more detailed advice.3% 


The Commission recommends engaging the Al 
Safety Commissioner (see Part C) to issue guidance 
on how to undertake a human rights impact 
assessment in this context. While the Commission 
has not identified a compelling case for HRIAs 
being compulsory for non-government bodies 

that engage in Al-informed decision making, that 
question should be considered further if necessary. 


Finally, there is a long-term trend that favours 
delivering government services and functions via 
government business enterprises, corporations and 
other non-government bodies. Thus, it is important 
that accountability not be diminished or lost when 
this results in those non-government bodies 
engaging in Al-informed decision making on behalf 
of the Australian Government. 


In Chapter 5, the Commission recommends 

an audit of all current or proposed use of Al- 
informed decision making by or on behalf of 
government. That audit (see Recommendation 4) 
should specifically include government business 
enterprises and non-government bodies that 
engage in decision making on behalf of the 
Australian Government. 


6.3 Transparency 


RECOMMENDATION 10: The Australian 
Government should introduce legislation 
to require that any affected individual 

is notified when a corporation or other 


legal person materially uses Alina 
decision-making process that affects the 
legal, or similarly significant, rights of the 
individual. 


The Commission uses the term ‘transparency’ to 
refer to people being made aware when Al is used 
in decision making that affects them. 


In Chapter 5, the Commission concluded that 
where the government engages in Al-informed 
decision making, it should do so transparently. In 
particular, the Commission recommended that 
the government notify any affected person where 
Al is materially used in making an administrative 
decision. 


The Commission considers this requirement should 
apply also to Al-informed decisions made by non- 
government entities. Knowing that a company 

uses Al to make decisions can be useful for people 
in deciding whether, or how, to engage with that 
company. 


(a) Notification of the use of Al 


The Discussion Paper proposed a legal requirement 
to notify an affected individual where Al is 
materially used in a decision that has a legal 

or similarly significant effect on the individual’s 
rights.3°8 Several stakeholders supported applying 
such a notification requirement to corporations and 
other non-government entities.*°° 
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Notification can be an important aspect of 
accountability in certain contexts, especially where 


human rights are engaged, and it was observed 
that notification requirements already exist in other 
contexts—for example, under Australian privacy 
legislation.4° 


Some stakeholders were wary of a notification 
requirement being imposed on business, 
particularly given the potential compliance costs 
involved.*°' It was submitted that exceptions may be 
required, with clarification sought regarding when 
the notification requirement would be triggered.*° 
Essentially, there was concern that notification 

only be required where the use of Al could have a 
significant effect on an individual, such as affecting 
their human rights. 


(b) Conclusion: Al notification requirement 


Like government, the private sector also should 

be required to notify the public of the use of Al 

in decision making that has a legal, or similarly 
significant, effect on people's rights. Such a 
requirement would help build public trust regarding 
the use of Al in ways that engage human rights. 


A notification requirement would also be a way 

of applying a key element of the Australian 
Government's Al Ethics Principles, which provide for 
transparent and responsible disclosure ‘to ensure 
people know when they are being significantly 
impacted by an Al system, and can find out when an 
Al system is engaging with them’. Research shows 
a clear public expectation that there be greater 
transparency in the use of Al by both the public and 
private sectors.4™4 


The Commission’s recommendation is supported 
by human rights law. The UN Guiding Principles 
on Business and Human Rights require clear 
communication from businesses about how their 
operations may impact on the human rights 

of individuals with whom they may come into 
contact.*°° The UN Special Rapporteur on freedom 
of expression has also concluded that the private 
sector should ‘make explicit where and how Al 
technologies and automated techniques are used 
on their platforms, services and applications’.*° 


Similar notification requirements are being 
considered in other jurisdictions. The Council 

of Europe Commissioner for Human Rights, for 
example, has called for governments to ensure 
that, when Al is used in ways that meaningfully 
affect human rights, this should ‘be made public 

in clear and accessible terms’. A draft law 

being considered by the New York City Council 
would require a company using an Al-assisted 
employment tool to assess and hire candidates for 
a job to disclose to candidates when such software 
is being used. 


The Commission acknowledges concerns raised 
by stakeholders about the potential breadth of 
the notification requirement, but considers that 
applying the requirement only to the material use 
of Al in decisions that have a legal, or similarly 
significant, effect constitutes a sufficient limitation. 
Where practical, this notification should take place 
before and after a decision is made. 


A notification requirement, such as that 
recommended here, would be similar to other 
obligations on business. For example, a similar 
burden applies to credit providers which have 
notification obligations under recent responsible 
lending laws. These are intended to equip potential 
customers with adequate information to enable 
them to make personal decisions regarding credit 
services offered to them.*°° 


6.4 Liability 


RECOMMENDATION 11: The Australian 
Government should introduce legislation 
that provides a rebuttable presumption 
that, where a corporation or other 

legal person is responsible for making 


a decision, that legal person is legally 
liable for the decision regardless of how 
it is made, including where the decision 
is automated or is made using artificial 
intelligence. 


Any system of accountability relies on clear lines of 
responsibility, and especially legal liability, for the 
consequences of decision making. 


In human rights law, it is important to be able to 
identify who is liable for a wrongful decision, so that 
an affected individual may pursue a remedy. This 
suggests that states, such as Australia, are obliged 
to ensure clear legal rules regarding liability for the 
use of Al-informed decision making.*"° 


The question of legal liability is usually, but not 
always, straightforward. As a general rule, whoever 
is responsible for making a decision is responsible 
also for any errors or other problems that arise in 
the decision-making process. Where this person 
has relied on a third party in the process of decision 
making, and the third party caused the error to take 
place, the first person remains liable for the error. 

It may be that the first person can seek redress 
from the third party, but that complexity should not 
be a barrier to an individual affected by the error 
seeking redress. That is, the affected individual 
should be able to seek redress, in the ordinary 
course of events, from the first person, not the third 


party. 


As with other forms of decision, usually the 
question of liability will be straightforward for 
Al-informed decision making. However, some 
complexities can arise—either where an Al- 
informed decision-making system operates largely 
autonomously, or where numerous parties are 
involved in designing, developing and using the 
system. 


Consequently, the Commission recommends the 
creation of a rebuttable presumption that legal 
liability for any harm that may arise from an Al- 
informed decision should be apportioned primarily 
to the legal person that is responsible for making 
the Al-informed decision itself. As this is no more 
than a presumption, this provision would anticipate 
exceptional situations where liability should be 
apportioned differently. 


The Commission considers such reform would 
more clearly apply the long-standing principle that 
underlies Australia’s law on legal liability. 


(a) Options for reform 


The challenges posed by Al-informed decision 
making to traditional concepts of liability were 
raised from the earliest stages of the Project 
consultation.*"! The Discussion Paper proposed 
reform to create a rebuttable presumption that the 
legal person who deploys an Al-informed decision- 
making system is legally liable for the use of the 
system.“ 


Several stakeholders supported the proposed 

new rebuttable presumption. They argued that 
legal liability for the use of an Al decision-making 
system is vital in achieving accountability, noting 
such a presumption would support transparency in 
decision making and would help to prevent harm.“ 


Other stakeholders opposed the Discussion 
Paper's proposal.*"* They argued that legislation is 
unnecessary, given existing law regarding liability, 
such as tort law and rules governing corporate 
disclosure.“ 


Whether for or against the proposal, stakeholders 
sought clarity that any such presumption be limited 
to entities that use, or have primary responsibility 
for the operation of, an Al-informed decision- 
making system.*'® Some stakeholders made 

clear that if a person is involved in developing 

an Al-informed decision-making tool, this should 
not necessarily result in that person being liable 

for how the tool is eventually used, especially if 
problems relate specifically to the deployment of 
the tool in ways that the developer of the tool could 
not reasonably have anticipated or addressed in 
advance. 


In short, if liability is to be shared among multiple 
parties, there should still be fair apportionment 
based on the level of responsibility for any errors or 
problems.*7 


(b) Conclusion: clarifying the law on liability 


There are extensive bodies of law on determining 
and apportioning legal liability for decisions that 
affect legal and similarly significant rights and 
interests. Those existing laws are likely to resolve 
most liability questions that arise in respect of Al- 
informed decision making. 
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Laws covering areas such as product liability and 
consumer Safety, negligence or discrimination are 
technology neutral, and apportion liability across 
different actors who bear legal responsibility 

for the outcome of the decision. For example, 

in the financial services sector, personal and 
organisational liability are expressly dealt with in 
legislation.*" 


However, this existing body of law is not 
comprehensive. Consequently, the Commission 
has concluded that there would be benefit in 
reform that addresses any situations of ambiguity 
regarding liability for Al-informed decision making. 


The Commission recommends reform that makes 
clear that ordinary liability principles generally 
should apply to Al-informed decision making. That 
is, there should be a rebuttable presumption that 
legal liability for any harm that may arise from 

an Al-informed decision should be apportioned 
primarily to the legal person that is responsible for 
making the Al-informed decision itself. 


There will be situations where this is inappropriate, 
so this should be no more than a general rule, or 
rebuttable presumption, which could be displaced 
if there are strong legal reasons for doing so. 
Legislation that makes this clear, along with 
guidance about how to apply this legal rule in a 
range of practical scenarios, is likely to assist in 
resolving many of the difficulties regarding liability 
in this context. 


6.5 Explainable Al and the right to 
a remedy 


RECOMMENDATION 12: Centres of 
expertise, including the newly established 
Australian Research Council Centre of 


Excellence for Automated Decision- 


Making and Society, should prioritise 
research on the ‘explainability’ of Al- 
informed decision making. 


RECOMMENDATION 13: The Australian 
Government should introduce legislation 
to provide that where a court, or 
regulatory, oversight or dispute resolution 
body, has power to order the production 
of information or other material from a 
corporation or other legal person: 


(a) for the avoidance of doubt, the 
person must comply with this order 


even where the person uses a form 
of technology, such as artificial 
intelligence, that makes it difficult to 
comply with the order 


(b) if the person fails to comply with 
the order because of the technology 
the person uses, the body may draw 
an adverse inference about the 
decision-making process or other 
related matters. 


As discussed in Chapter 5, the problem of ‘opaque’ 
or ‘black box’ Al arises where the use of Al obscures 
the rationale or reasons for a decision. This can 
frustrate any legal right to reasons, which can in 
turn make it difficult or impossible to challenge the 
merits or lawfulness of the decision itself. In human 
rights terms, this can deny a person's right to a 
remedy. 


Australian law generally entitles a person affected 
by an administrative decision to reasons for 

the decision. However, it is only in exceptional 
situations that a person would be legally entitled to 
reasons for a decision made by a non-government 
entity. Nevertheless, opaque or black box Al can 

be a significant problem wherever it arises—in 
government and the private sector. 


This section of the chapter addresses this issue in 
two ways. 


First, it is good practice to provide reasons for 
decisions that affect a person’s legal or similarly 
significant rights, regardless of the status of the 
decision maker and even where there is no legal 
requirement to provide reasons. The Commission 
recommends further research on explainable Al, 
and expert guidance from government on how to 
provide reasons for Al-informed decisions. 


Secondly, the Australian Government should 
introduce legislation to address the problem of 
black box Al and ensure that people are able to 
access a remedy for unlawful Al-informed decisions 
by the private sector. In particular, the Commission 
recommends legislation be introduced to make 
clear that where a court or other regulatory body 
orders a person (eg, a corporation) to produce 
material, this order cannot be refused simply 
because of the form of technology, such as Al, that 
the person uses in its decision-making and other 
processes. 


(a) A statutory right to reasons? 


In some exceptional situations, a non-government 
entity may be legally required to provide reasons 
for certain categories of decisions. For example, 

a telecommunications provider must provide a 
consumer with reasons where it decides to restrict 
supply of, or access to, a telecommunications 
service.*!? In addition, in some industries or sectors 
it may be considered good practice to provide 
reasons for certain decisions. 


Beyond these situations, the Discussion Paper 
raised the possibility of creating a new statutory 
right to reasons for Al-informed decisions made 

by non-government entities.47° This would be a 
significant reform, involving a major departure 
from the current position that, generally speaking, 
decisions by non-government bodies do not carry a 
legal entitlement to reasons. 


There was opposition to such a reform on three 
principal grounds.*' The first was technical: some 
Al-informed decision-making systems were said 

to be too complex or sophisticated to generate a 
useful explanation.*” Even if this problem could 
be overcome, there was concern that doing so 
would be unreasonably costly, both financially and 
due to the time it would take for an organisation 


to generate an explanation, an issue particularly 
relevant to small and medium-sized enterprises.*? 


The second objection was that a right to reasons 
could cause unintended consequences. For 
instance, this could breach privacy law by revealing 
personal information used in the decision-making 
process,**4 which in extreme cases could be 
exploited by third parties for criminal or other 
improper purposes.**° Giving reasons could also 
reveal commercially sensitive information related to 
companies’ decision-making processes.4?6 


The third objection was that a new general right to 
reasons would be a disproportionate response to 
the problem of black box Al. For example, it was 
suggested that where there was a demonstrated 
need for a right to reasons for a specific type of 
decision, reform could be targeted to that need.*?’ 


On the other hand, some stakeholders expressed 
support for a more general right to reasons— 
essentially as a way of addressing the power 
imbalance experienced by people who are 
negatively affected by Al-informed decisions 
made by corporations.® While the technical, 
commercial and other problems referred to above 
were acknowledged, practical solutions were 

also suggested. For example, limited disclosure 
of reasons for a decision could be made if an 
explanation risks exposing information that 

is sensitive for commercial, privacy or other 
reasons.” 


If law reform were not pursued, it was noted that 
guidance could be helpful in advising the private 
sector on how to provide reasons for Al-informed 
decisions.“ 


(b) Legal restrictions on opaque decisions? 


When it comes to the explainability of Al-informed 
decision making, the Commission distinguishes 
between two related, but different, concepts: 


e an obligation to provide reasons, which 
involves an active process of explaining the 
basis or rationale of a decision 
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e an obligation not to engage in opaque 
decision making, which means that the 
decision and the method by which it 
was made would be incapable of being 
scrutinised to discern key aspects of how 
and why it was made. 


The Discussion Paper flagged the possibility of 
reform to address the second of these concepts. 


Some stakeholders supported this general 

idea, on the basis that this was an important 
protection against unaccountable human rights 
infringements.**' One stakeholder argued that such 
an obligation would not go far enough, in that it 
would not be equivalent to a right to reasons.* 


Other stakeholders opposed, or offered only 
qualified support, for the creation of this sort of 
obligation.**? Several stakeholders questioned the 
legal policy basis for rendering private decisions 
unlawful, or for imposing a higher threshold 

on private sector decisions when reasons are 
required for so few categories of decision.**4 Some 
considered creating an obligation not to engage 
in opaque decision making would violate the rule 
of technological neutrality,*° and that this would 
be a disproportionate response that would inhibit 
innovation.*%¢ 


(c) Research on explainable Al 


‘Explainable Al’ is an emerging field of research, 
with experts and others in Australia and overseas 
calling for more work to be done in this area.*?’ 


The Commission acknowledges that there are 
significant challenges in designing and developing 
Al-informed decision-making systems that can yield 
reasons or explanations for the decisions that they 
produce. For this reason, the Discussion Paper 
called for research in this area to be prioritised.*° 


There was strong support for this proposal.*?° 
Stakeholders made some suggestions for how 

it could be implemented. For example, it should 
take account of existing research in this area, with 
a suggestion that grants not be limited to centres 
of expertise, and that funding be allocated to 
prioritise interdisciplinary research programs, so 
that legal and scientific objectives can be pursued 
holistically.4° 


(d) Conclusion: supporting explainable Al and 
the right to a remedy 


A human rights approach requires access to an 
effective remedy where an individual’s human 
rights have been breached. 


(i) Non-government decision making 


Unlike administrative decisions made by 
government, there are limited circumstances where 
an affected individual may be legally entitled to 
reasons for a decision by a private sector body. 


There are some exceptions. For example, in some 
situations a telecommunications provider must 
explain a decision to a consumer, such as where 

a restriction is placed on supply of or access 

to a telecommunications service at the time of 

the application, a supplier must explain to the 
consumer the general nature of the reasons for the 
restriction at the time of application.’ 


Some stakeholders suggested, rightly in the 
Commission’s view, that there may be a case for 
further exceptions. That is, there may be other 
non-government decision making where reform 
is warranted to create a legal entitlement to 
reasons.” 


Where a non-government entity is legally required 
to provide reasons for a decision, this rule should 
apply regardless of how the decision is made, 
including where Al (especially automation) is used in 
the decision-making process. As a corollary, if there 
is currently no legal obligation to provide reasons, 
the use of Al in the decision-making process does 
not necessarily mean that a new obligation to 
provide reasons should be created. There are 
genuine commercial and technical obstacles to 
providing an explanation for Al-informed decisions. 


The Commission applies the same general 
approach to administrative decisions in Chapter 
5. There are thus two situations that require close 
attention, when it comes to the explainability of 
Al-informed decision making by non-government 
bodies. 


The first is where a company or industry is already 
subject to a requirement to provide reasons for 
particular types of decision—either because this 
requirement is imposed by law or it applies via 


a voluntary or similar code. Such arrangements 
should continue to apply to Al-informed decision 
making. 


The second situation relates to regulation, oversight 
and dispute resolution. Regulators, tribunals, 
courts and others commonly have powers to 
obtain information from non-government entities 
to fulfil their functions in this area. These powers 
to obtain information are often used in order to 
shed light on how or why a decision was made. 
Several regulators—including the Australian 
Competition and Consumer Commission (ACCC),4*4 
the Australian Securities and Investments 
Commission (ASIC)**° and the Office of the 
Australian Information Commissioner (OAIC)#4°— 
have compulsory information-gathering powers, 
with penalties for non-compliance. 


Such powers can involve the compulsory 
production of material to a regulator or dispute 
resolution body, or they can involve the provision 
of material by one party in a dispute to one or 
more other parties. For example, courts regularly 
supervise discovery and related processes in 
litigation, whereby one party is required to provide 
information relating to the dispute to another party. 


Regulatory, oversight and dispute resolution 
bodies, including courts, rely on these powers to 
obtain the evidentiary material they need to fulfil 
their statutory, and sometimes constitutional, 
functions—namely to determine the true state of 
affairs in a given scenario and, ultimately, to uphold 
the law. However, the advent of opaque or black 
box Al means that those powers and functions 
could be frustrated where non-government entities, 
which are subject to the jurisdiction of these 
regulatory, oversight and dispute resolution bodies, 
opt to make decisions in a way that is incapable of 
providing the information those bodies need. 


These bodies often focus their attention especially 
on areas of decision making that have serious 
human rights implications, such as consumer 
protection, workplace relations and access to 
services. For these reasons, there is a strong public 
interest in making clear that opaque or black box 
Al is unacceptable in decision making that affects 
people’s legal and other similarly significant rights. 


The Commission thus recommends that the 
Australian Government introduce legislation 

to preserve the vital functions and powers of 
regulatory, oversight and dispute resolution bodies, 
including courts. Specifically, the Government 
should make clear that where such a body orders 

a person (eg, a corporation) to produce material, 
this order cannot be refused simply because of the 
form of technology, such as Al, that the person uses 
in its decision-making and other processes. 


Moreover, if the person fails to comply with the 
order because of the technology the person 
chooses to use, the body should be able to draw 

an appropriate adverse inference about the 
decision-making process and other related matters. 
In regulatory, oversight or dispute resolution 
processes, such a negative inference makes it more 
likely that the body will ultimately find against the 
person that has failed to comply with the order to 
produce material. 


(ii) Research into explainable Al 


There is a trend towards building an explanation 
function into Al-informed decision making 

systems, including on the part of leading software 
companies.*” Nevertheless, building an explanation 
into an Al-informed decision-making process can be 
technically difficult and expensive. 


In the United States and elsewhere, there has 

been an increase in funding for research that aims 
to ensure Al-informed decision making can be 
explained.* Following the establishment of the 
Australian Research Council Centre of Excellence for 
Automated Decision-Making and Society in October 
2019, Australia now has a similar opportunity. This 
Centre aims to formulate world-leading policy and 
practice and inform public debate, and to create the 
knowledge and strategies necessary for responsible 
and ethical automated decision-making.“ 


The Centre has already conducted collaborative 
research in these areas. The Commission supports 
that approach to research in this area, where 
multi-disciplinary collaboration can bring together 
different professional and academic perspectives. 
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7.1 Summary 


This chapter considers a range of government policy, co- and self-regulatory measures. 
These measures are important because, while formal regulation through legislation is 
important, it cannot be the sole means of promoting human rights and accountability in 
the use of Al-informed decision making. 


The recommendations in this chapter are largely preventative, in the sense that they are 
designed to prevent problems, and especially human rights violations, from occurring.*°° 


The measures recommended by the Commission include: 
* standards and certification, typically considered at the design stage of Al-informed 
decision-making systems 


* regulatory sandboxes and human rights impact assessments, including public 
consultation at the development stage 


* rules for government procurement of Al decision-making systems, at the 
deployment stage 


* the role of human oversight and intervention in the monitoring and evaluation 
stage. 
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72 The role of co-regulation and 
self-regulation 


Co- and self-regulation—sometimes known as ‘soft 
law'—commonly include measures such as codes 
of practice, guidelines, directives, industry or sector 
rules, protocols and other guidance.**! 


Self-regulation refers to when an organisation or 
a group of similar organisations 


voluntarily develops, administers and enforces 
its own solution to address a particular issue, 
and where no formal oversight by the regulator 
is mandated. Self-regulatory schemes are 
characterised by the lack of a legal backstop to 
act as the guarantor of enforcement.‘ 


Numerous examples of self-regulatory measures 
have emerged from the tech sector in recent years. 
As discussed further below, ‘Al ethical frameworks’ 
are a particularly common form of self-regulation in 
this area. 


Co-regulation generally refers to a process of 
collaboration—usually where an industry, sector or 
company works with government—in 


developing, administering and enforcing a 
solution, with arrangements accompanied by 

a legislative backstop. Co-regulation can mean 
that an industry or professional body develops 
the regulatory arrangements, such as a code of 
practice or rating schemes, in consultation with 
government. While the industry may administer 
its own arrangements, the government provides 
legislative backing to enable the arrangements 
to be enforced.*°? 


Several Australian Government regulators 

have expertise in co-regulation. The Australian 
Communications and Media Authority (ACMA) 
has identified the optimal conditions for 

the development of co-regulation, including 
environmental factors, such as the competitiveness 
of the market where co-regulation is being 
developed, and regulatory factors, such as 
stakeholder participation in the development 

of the scheme and cooperation with the 
regulator.494 The ACMA is currently overseeing the 
development of a voluntary code to address the 


issue of misinformation across digital platforms, 
as recommended by the ACCC's Digital Platforms 
Inquiry final report.*°> 


A common benefit associated with self- and co- 
regulation is that they can be more flexible or 
adaptable to a changing environment, as compared 
with more conventional regulation via legislation. 
Stakeholders observed that this is particularly 
important in the context of Al, which is developing 
rapidly, and so such forms of regulation can 

set rules while also supporting innovation and 

the development of the digital economy.‘ On 

the other hand, the majority of stakeholders 
considered self-regulation should not be the sole 
means of protecting human rights in this area, and 
argued instead for self- and co-regulation to be an 
addition—rather than replacement—for stronger 
regulatory measures in law.*9” 


N 


(reliable 


Human Rights and Technology Final Report * 2021 ° 87 


88 


7. Encouraging better Al-informed decision making 


7.3 Al ethical frameworks 


In recent years, it has become common to refer 

to risks of human harm, associated with the 
development and use of Al, as ethical issues. 

For example, risks that Al might be used in ways 
that breach the right to privacy or equality are 
frequently referred to as ethical risks, even though 
they often have important legal implications. 


A large number of self-regulatory initiatives— 
framed around ethical risks associated with Al— 
have been developed in response. Governments, 
private sector organisations and representative 
bodies, as well as civil society, have all developed 
ethical policies and principles that aim to guide 
the development and use of Al. This Final Report 
refers to such initiatives collectively as ‘Al ethical 
frameworks’. 


As explored in this section of the chapter, there 

are real questions about the effectiveness of many 
existing Al ethical frameworks in achieving their 
purpose—to guide companies, researchers and 
others involved in developing and using Al, to avoid 
harm. 


To be effective in achieving its legitimate aims, 
the Commission considers that any Al ethical 
framework should: 


1. not be treated as a substitute for the law. 
Rather, it should supplement, complement 
and reinforce the law 


2. be consistent with international human 
rights law 


3. be framed in a way that provides useful, 
practical guidance, especially for those who 
develop and use Al. 


(a) The rise of Al and ethics 


Ethical principles or rules tend to be self-regulatory. 


They guide the person responsible for a decision 
to make better decisions. Generally, ethics are not 
legally binding, although in some highly-regulated 
professions or industries such as medicine or law, 
they can be codified into a set of rules that are 
legally enforced. 


Some Al ethical frameworks apply exclusively to 
Al, and others to a narrower or broader range of 
technologies and techniques. They can also apply 
broadly to all uses of Al, or they can be directed 
towards a particular area, such as the use of Al in 
healthcare. 


Generally, an Al ethical framework is premised 

on an understanding of the particular risks of 
human harm that arise in a particular field of 
operation. Different Al ethical frameworks adopt 
different approaches in understanding risks of 
harm.*58 Sometimes that understanding draws 

on bodies of law, such as human or consumer 
rights; sometimes it draws on areas of developed 
research or thinking, such as consequentionalist or 
teleological ethics; sometimes the understanding is 
more idiosyncratic, in the sense that it reflects the 
views and experience of the people who created 
the Al ethical framework in question. 


Al ethical frameworks generally offer guidance and 
so are not legally binding. While it is possible for a 
legislature to incorporate an ethical framework into 
legislation, this has been rarely done in respect of 
ethics and Al. 


In Australia, the most prominent Al ethical 
framework is the Australian Government's Al Ethics 
Principles (see Box 4).4°° There are also a range of 
industry-led Al ethical framework initiatives, such as 
Microsoft's Al Principles.*°" 


The value of Al ethical frameworks lies in the extent 
to which they assist in identifying and addressing 
risks to humans in respect of Al-informed decision 
making. The Discussion Paper observed that, 

while Al ethical frameworks were a relatively new 
phenomenon, independent assessments of existing 
frameworks tended to cast down on their value 
and effectiveness.’ As a result, the Commission 
sought stakeholder feedback on the place of Al 
ethical frameworks, and proposed a more detailed 
independent inquiry into these frameworks. 


Box 4: Australian Government Al Ethics Principles*© 
In November 2019, the Australian Government released the A/ Ethics Principles, which aim to ‘foster 
public trust and enable adoption and development of Al’.* The Principles are: 


* Human, social and environmental wellbeing: Throughout their lifecycle, Al systems should 
benefit individuals, society and the environment. 


Human-centred values: Throughout their lifecycle, Al systems should respect human rights, 
diversity, and the autonomy of individuals. 


Fairness: Throughout their lifecycle, Al systems should be inclusive and accessible, and 
should not involve or result in unfair discrimination against individuals, communities or 
groups. 


Privacy protection and security: Throughout their lifecycle, Al systems should respect and 
uphold privacy rights and data protection, and ensure the security of data. 


Reliability and safety: Throughout their lifecycle, Al systems should reliably operate in 
accordance with their intended purpose. 


Transparency and explainability: There should be transparency and responsible disclosure 
to ensure people know when they are being significantly impacted by an Al system, and can 
find out when an Al system is engaging with them. 


Contestability: When an Al system significantly impacts a person, community, group or 
environment, there should be a timely process to allow people to challenge the use or output 
of the Al system. 


Accountability: Those responsible for the different phases of the Al system lifecycle should 
be identifiable and accountable for the outcomes of the Al systems, and human oversight of 
Al systems should be enabled. 


The Commission has received mixed feedback on The most common concerns regarding existing Al 
the value and effectiveness of Al ethical frameworks ethical frameworks related to: 


to minimise harm to humans. f ; 
* the idea that Al ethical frameworks could 


Some stakeholders saw benefit in at least some serve as an alternative to binding laws that 
forms of Al ethical framework, especially where protect human rights and other important 
the framework is grounded in human rights law, or interests 


it can allow regulation that is faster to adapt than 

ordinary legislation tends to be. However, many 

more stakeholders expressed strong scepticism 

about the effectiveness of existing Al ethical 

frameworks, especially those adopted by private the fact that a focus on ethics was 

companies.* inherently more vague and subjective than 
focusing on applying existing international 
and domestic human rights law.*°” 


the lack of generally agreed normative 
content to provide the foundation of Al 
ethical frameworks 
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Several stakeholders did not support an 
independent inquiry into Al ethical frameworks. 
There was concern that this would distract from 
the more pressing task of applying binding laws 
to the use and development of Al, especially in 
decision making that affects people's legal or 
similarly significant rights.4°® Stakeholders also 
suggested drawing on existing work, here and 
overseas, assessing the effectiveness of Al ethical 
frameworks.*©° 


Other stakeholders did see utility in such an 
inquiry.*”° They saw an inquiry as a chance to 
consolidate and harmonise the various initiatives 
in this area with a view to providing greater clarity 
and certainty of expectations.4”! This problem was 
particularly acute for multi-national corporations, 
which might be subject to a very large number of 
Al ethical frameworks.472 


(b) Conclusion: Al ethical frameworks and 
regulation 


The Commission acknowledges that Al ethical 
frameworks could play a valuable role in promoting 
human rights and accountability in respect of 
Al-informed decision making. However, many Al 


ethical frameworks have had limited positive impact 


in protecting human rights and addressing broader 
harms. 


Recent research and evaluation suggest that some 
terms commonly used in Al ethical frameworks 
are vague or subject to differing interpretations. 
Too often, there is a wide gap between the 
principles espoused in Al ethical frameworks 

and how Al-powered products and services are 
actually developed and used. This can contribute 
to irresponsible and damaging use of Al.4”? Several 
experts have rejected self-regulatory ethical 
frameworks as a sufficient tool to protect human 
rights in the context of Al-informed decision 
making, advocating instead for more robust, 
enforceable accountability mechanisms, including 
those set out in law.*” 


The value of an Al ethical framework is primarily as 
a tool of self- or co-regulation, to assist those who 
develop and use Al to identify and address risks 

to people, especially in the context of Al-informed 
decision making. To achieve this goal, the following 
general principles should be applied. 


First, an Al ethical framework should not be a 
substitute for the law. Rather, as in all other areas 
of activity, Al-informed decision making should be 
governed primarily by /aw. The starting point should 
be to apply our current laws more effectively in 
respect of Al-informed decision making, and to 
identify gaps in the law that require reform. 


The legitimate role of an Al ethical framework is to 
supplement, complement and reinforce the law, 
by filling gaps where detailed legal regulation is 
inappropriate and by providing ethical guidance 
where the law does not, and should not, articulate 
a clear rule to follow. 


The presence of an Al ethical framework cannot 
justify a government agency or company acting 
unaccountably, leaving affected individuals with 
no way of challenging an Al-informed decision or 
enforcing their rights. 


Secondly, Al ethical frameworks should be 
consistent with international human rights law. 
Human rights law sets out agreed norms, as well as 
an extensive body of jurisprudence demonstrating 
how those norms should be applied in concrete 
situations. By contrast, current ethical frameworks 
have no equivalent agreed normative foundation.*”° 


The importance of working from an agreed 
normative starting point is especially important 
given that many leading tech companies operate 
across multiple jurisdictions. A human rights 
approach rests on ‘a level of geopolitical recognition 
and status under international law that no newly 
emergent ethical framework can match’.*”° As the 
UN High Commissioner for Human Rights observed 
in 2019, there is no need to ‘start from scratch’ in 
this area, given the strength of human rights to 
address the challenges posed by new and emerging 
technologies.*’” 


Thirdly, an Al ethical framework should be 
practically useful, especially for those who develop 
and use Al. A legitimate criticism of many existing Al 
ethical frameworks is that they tend to be framed 
at a high level of abstraction or generality. For 
example, the principle ‘do no harm’ is common in 
such frameworks, but its precise meaning in the 
context of developing and using Al is not widely 
understood or agreed. 


Such vague principles do not provide the level 

of specificity that an engineer, data scientist or 
someone else intimately involved in creating an 
Al-informed decision-making tool would need to 
ensure it is designed, developed and deployed in 
a way that protects human rights. 


In its 2020 review of 47 ethical frameworks from 
across the world, Harvard University’s Berkman 
Klein Center observed ‘a wide and thorny gap 
between the articulation of these high-level 
concepts and their actual achievement in the 

real world’.4”8 Similarly, a 2018 study found that 
expressly instructing software engineering graduate 
students and professional software developers to 
consider ACMA’s Code of Ethics in their decision- 
making ‘had no observed effect’.4”° 


Instead, both the content of an Al ethical 
framework, and the way that content is 
communicated to the people who are responsible 
for applying the framework, need to reflect a 

real, practical understanding of how Al-powered 
products and services are developed. Among other 
things, this involves a broad, collaborative approach 
in developing Al ethical frameworks. As Gradient 
Institute has observed: 


The people ultimately responsible for an Al system 
must also be responsible for the ethical principles 
encoded into it. These are the senior executives 
and leaders in business and government who 
currently make strategic decisions about their 
organisation's activities. The ethical principles that 
they determine will, as with the decision-making 
they already do, be informed by a mixture of their 
organisation's objectives, legal obligations, the 

will of the citizens or shareholders they represent, 
wider social norms and, perhaps, their own 
personal beliefs.*%° 


The oversight of Al ethical frameworks, and 
guidance on applying principles such as those set 
out immediately above, should be within the remit 
of the Al Safety Commissioner (Recommendation 
22). While recent research on Al ethical frameworks 
has reduced the need for a comprehensive 
independent inquiry in this area, it is appropriate 
for this proposed new body to have the power to 
review specific Al ethical frameworks from time to 
time.**" 


74 Human rights by design and Al 


The Commission supports applying ‘human rights 
by design’ to Al. This involves systematically 
embedding human rights at every stage of the 
design, development, deployment and ongoing 
monitoring of products, services and systems that 
use Al.4°2 


Human rights by design is a developing field, and is 
generally considered to include a number of core 
elements (see Box 5). 
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Box 5: Human rights by design 
approach 


The Oxford Handbook of Al Ethics described 
four central pillars in applying human 
rights by design to ‘Al systems’, which 
includes Al-informed decision-making 
systems: 


* Design and deliberation: systems 
should be designed to comply with 
international human rights law. 

The design process should include 
appropriate mechanisms such as 
public consultation for Al systems 
posing a high risk to human rights, 
which may lead to redesign or the 
incorporation of mitigation strategies 


Assessment, testing and evaluation: 
regular testing for human rights 
compliance from formulation of the 
initial proposal, through to design, 
development, prototyping and real- 
world implementation. Ongoing 
monitoring and periodic review 
should follow once the Al system has 
been deployed. 


Independent oversight, investigation 
and sanction: an independent, 
external, properly resourced 

and technically competent body 
should be established to oversee 
human rights compliance of the Al 
system, within a legally mandated 
institutional structure. 


Traceability, evidence and proof: 

Al systems should be designed 

to secure auditability, such that 

the system can be subject to a 
meaningful review and demonstrate 
ongoing human rights compliance.*®? 


As stakeholders observed,** human rights by 
design often builds on and complements similar 
design-led approaches, especially: 


e Privacy by design, which aims to ensure 
privacy is built ‘into projects from the design 
stage onwards’, and should be considered 
‘a fundamental component of effective 
data protection’. Privacy by design involves 
‘tak[ing] steps at the outset of a project that 
minimise risks to an individual's privacy, 
while also optimising the use of data’.4®° 


Safety by design, a set of principles 
developed by the Australian eSafety 
Commissioner, which guide companies to 
use a design process to assess, review and 
embed user safety into online services, 
providing an achievable voluntary standard 
for private industry.‘ 


There was strong support among stakeholders for 
encouraging human rights by design in research 
and development using Al, with stakeholders seeing 
benefit in embedding human rights protection 

at the earliest stages of the Al life cycle.*°” 
Stakeholders encouraged wide consultation with a 
range of experts, decision makers and with those 
who will be impacted by the use of Al in decision 
making.‘ Human rights by design could also be 
applied through algorithmic or human rights impact 
assessments.*°? 


There was a recognition that Australia would 
benefit from a process, led by an expert taskforce, 
to develop the human rights by design concept 
further, and make it easier to apply to the 
development and use of Al in decision making.*”° 


The Commission encourages those responsible 

for designing, developing and overseeing Al- 
informed decision-making systems to adopt 

a human rights by design approach from the 
earliest stages of considering and developing such 
systems. This approach can help government and 
non-government bodies to identify and address 
human rights issues before they result in rights 
breaches. It can also support and reinforce the legal 
mechanisms discussed in Chapters 5 and 6. 


While human rights by design is still in its relative 
infancy, it is similar to other design-led approaches 
that can drive positive results. There are also 
parallels with approaches like human rights due 
diligence, which has been used by the private 
sector to assess the human rights impacts of their 
operations across the supply chain.**" 


As set out in Recommendation 14 below, the 
Commission recommends practical steps to 
embed the principles of human rights design in the 
development of Al-informed decision making. 


75 Design standards and 
certification 


Voluntary standards and certification schemes can 
help to promote human rights and accountability, 
especially in the design of Al-informed decision- 
making systems. 


Standards set out ‘specifications, procedures and 
guidelines that aim to ensure products, services, 
and systems are safe, consistent, and reliable’.4 
A standard can apply to particular groups or 
industries, with some standards operating within 
a national or sub-national jurisdiction or more 
broadly across multiple nation states. Standards 
tend to be voluntary, but like ethical frameworks, 
standards can be given the force of law, through 
legislation. 


The Commission recommends (see 
Recommendation 14) that a multi-disciplinary 
taskforce consider ways to develop and apply 
voluntary standards and certification schemes for 
Al. 


(a) Experience in Australia and overseas 


There is already a substantial body of work 
regarding global standards for Al. This includes 

the work of professional bodies, such as the IEEE’s 
guide to ‘ethically aligned design’ for autonomous 
and intelligent systems, and international 
organisations, such as the OECD's Principles on Al.*°? 
The European Commission has also considered 
how the EU's existing safety and product liability 
regulatory framework, including product standards, 
apply to Al, loT and robotics.‘ 


The Australian Government's 2020 Al Action 

Plan states that ‘standards will be a key enabler 

in helping businesses confidently adopt Al 
solutions’. While the Government has not 
incorporated any Al standards into Australia law 
or policy, in 2019, it provided funding to Standards 
Australia, to consult on the development of 
standards to Al.*%° The resultant A/ Standards 
Roadmap sets out steps to support the 
development of standards for Al in Australia.‘ 


The Government's Al Ethics Principles also contains 
general principles for designing, developing, 
integrating or using Al.4°8 


Stakeholders recognised that voluntary standards 
offer flexibility in protecting human rights, and they 
can be developed with the private sector as a co- 
regulatory measure. Some argued that standards 
could help promote design of Al-informed decision- 
making tools that bring human rights to the fore, 
especially when standards have legal force.” 


Standards share some common features with 
voluntary certification or ‘trustmark’ schemes. 
These schemes tend not to be legally binding, 
but they can incentivise compliance, especially by 
influencing consumer choice. The EU’s GDPR, for 
example, encourages Member States to establish 
‘data protection certification mechanisms’ and 
‘data protection seals and marks’ to demonstrate 
compliance with the Regulation.” 


Stakeholders expressed interest in the use of 
trustmarks to certify products and services using Al 
as ethically or human rights compliant.” Several 
stakeholders specifically referred to the idea 
advanced by Australia’s then Chief Scientist, Dr Alan 
Finkel, to establish a ‘Turing Stamp’—a proposed 
voluntary certification scheme for ‘ethical Al’ that 
would be independently audited. 


The OAIC has supported a certification scheme as 
a means of ensuring that individuals are provided 
with information about the privacy credentials 

of third parties, such as businesses, they interact 
with.° 


There was some opposition to the idea of a 
certification scheme applying to Al generally; 
rather, certification should be sector specific and 
proportionate to the risk of identified harms.°°° 
One stakeholder cautioned against any kind 
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of certification scheme, noting the difficulty 

of obtaining a sufficient level of expertise to 
govern such a scheme, and questioning whether 
certification would lead to better decision making 
by the public.°°% 


There was support among stakeholders for a 
multidisciplinary taskforce, informed by public 
consultation, to advance Australia’s approach to 
standards and certification schemes in the area of 
Al generally, and Al-informed decision making in 
particular.°°” 


(b) Conclusion 


The Commission supports the development of 
voluntary design standards and certification 
schemes in respect of Al-informed decision making. 
Such initiatives can help bring human rights issues 
to the fore at the early stages of the Al life cycle, 
and is supported by the UN Guiding Principles on 
Business and Human Rights.°°% 


In its current review of federal privacy law, 

the Australian Government is considering the 
‘desirability and feasibility of an independent 
certification scheme to monitor and demonstrate 
compliance with Australian privacy laws’.°°? This 


is a good idea. However, any such certification 
scheme should take into account all human rights 
that are engaged by Al-informed decision making. 


In the Commission's view, a multi-disciplinary 
taskforce (see Recommendation 14) should 
consider: 


e how to develop human rights compliant 
standards for Al-informed decision making, 
building on the Australian Government's Al 
Action Plan 


e whether these standards, guided by human 
rights, should be given legislative force 


e whether a self- or co-regulatory certification 
scheme should be established in this area 
and, if so, how the scheme should operate 


* how to ensure standards and certification 
are integrated into international processes, 
given recommendations in the Standards 
Australia A/I Roadmap, referred to in the 
Government's Al Action Plan.>"° 


The taskforce should consult with a range of 
experts, both here and overseas, across a range 
of disciplines, including human rights, law and 
engineering. 
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7.6 Regulatory sandboxes 


RECOMMENDATION 14: The Australian 
Government should convene a multi- 
disciplinary taskforce on Al-informed 
decision making, led by an independent 
body, such as the Al Safety Commissioner 
(Recommendation 22). The taskforce 
should: 


(a) promote the use of human rights 
by design in this area 


(b) advise on the development and 
use of voluntary standards and 
certification schemes 


(c) advise on the development of 
one or more regulatory sandboxes 
focused on upholding human rights 
in the use of Al-informed decision 
making. 


The taskforce should consult widely in the 
public and private sectors, including with 
those whose human rights are likely to 

be significantly affected by Al-informed 
decision making. 


A regulatory sandbox is a controlled environment, 
created by a regulator, to enable people to test new 
products, services or practices under the regulator's 
supervision. 


Regulatory sandboxes are especially popular in 

the context of new and emerging technologies. 
They aim to facilitate dialogue with policy makers, 
and enable regulators ‘to try out new rules and 
observe their impact on the technology in an 
environment where wider damage or danger to the 
public is limited'.*"! Experience from the operation 
of the sandbox can be used to inform policy and 
regulation." 


Regulatory sandboxes can provide mutual benefit 
to both the regulator and participant, including: 


e reducing regulatory uncertainty for 
innovators 


* providing an opportunity to discuss, in 
confidence and candidly, the potential uses 
of new technologies 


* obtaining early warning that a feature of 
a new product may not be acceptable, 
allowing that feature to be modified.” 


On the other hand, some criticise the regulatory 
sandbox concept because it: 


* can create a risk or perception that 
regulators inappropriately support or 
favour certain tech start-ups 


* may create limited safeguards against harm 
to individuals 


* can inappropriately lower barriers to entry 
in areas where the actual risk of harm to 
humans is high." 


In the specific context of Al, there can be a tension 
between the need to be candid with a regulator, 
and the commercial sensitivity of an opaque 
algorithm. 


Noting this complexity, the Commission sees 
benefit in trialling regulatory sandboxes for Al- 
informed decision making, as a way of promoting 
responsible innovation. This chapter recommends 
expert advice be commissioned on the use of 
regulatory sandboxes for Al-informed decision 
making. 


(a) Options for reform 


Some see regulatory sandboxes as a way of 
supporting innovation, while enabling regulators 

to test new regulatory models. To date, regulatory 
sandboxes have been deployed mostly in the 
‘FinTech’ area—that is, in respect of technology that 
enables or supports banking and financial services. 


In Australia, ASIC is running an ‘enhanced 
regulatory sandbox’ that exempts the provider 

of an eligible financial product, service or credit 
activity from relevant licensing requirements for 
up to 24 months while the entity participates in the 
sandbox.°'® 


Other jurisdictions have begun to establish 
regulatory sandboxes in areas more closely aligned 
with human rights, such as the protection of data 
privacy (see Box 6.) 
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Box 6: UK Information Commissioner's 
Office regulatory sandbox 


In 2019, the UK's ICO introduced 

a regulatory sandbox to support 
organisations developing products 
and services that ‘use personal data in 
innovative and safe ways and ... deliver 
a potential public benefit’.*" 


In 2020, the ICO published a report 

on the participation of the National 
Health Service’s NHS Digital, which was 
developing a central consent mechanism 
for patients to agree to share their 
personal data, including with researchers 
working on a COVID-19 vaccine.°"” 

The NHS sought advice from the ICO 
regarding the service, including the 
project's privacy impact, issues regarding 
notification and consent, compliance 
with the GDPR and UK data protection 
legislation. 


The regulatory sandbox process was 
considered to be of mutual benefit to 
the ICO as regulator and the NHS. The 
ICO gained a better understanding of the 
sharing of patient data and the interaction 
of privacy and other related laws and 
policies. The NHS, developing the service 
under tight timescales, received expert 
advice that gave it confidence it would 
respect privacy and related rights and 
maintain public trust.°"8 


A key area of focus for the 2020-21 
regulatory sandbox for the ICO is the 
development of products or services 
‘that support complex data sharing in 
the public interest’, but where the data 
sharing may pose a high risk to the 
public and information rights, involve the 
processing of personal data, or use novel 
or innovative techniques.°'? 


Stakeholders identified a range of potential benefits 
of using regulatory sandboxes to protect human 
rights, including: 


fostering a more open engagement 
between regulators, government agencies 
and the private sector, particularly small to 
medium-sized enterprises?” 

the potential for early identification of risks 
or breaches of regulation, which can save 
costs associated with products and services 
that are unviable° 

the early identification of risks or rule 
breaches can also be used to improve the 
regulatory system itself”? 

the building of public trust in Al that has 
been subjected to this adapted form of 
regulation." 


Stakeholders encouraged a broad view of what 
could be tested in a regulatory sandbox, including 
not only new technology products but also 
methods of review and evaluation.” Stakeholders 
also pointed to the importance of learning from 
sandboxes that have already been established, 

or suggested that human rights be added as a 
measurement to existing sandboxes.°° 


Stakeholders commented on the parameters of the 
sandbox, including in relation to: 


e its focus, such as the testing of an Al- 
informed decision-making system for 
algorithmic bias and fairness, measuring 
compliance of an Al product or service 
with data access and sharing rules, or the 
consideration of specific technologies such 
as facial recognition technology or self- 
driving vehicles 

eligibility, including that it be an opt in, with 
no incentive offered other than the benefits 
of co-regulatory collaboration 

applicable regulation, including assessing 
adherence to domestic and international 
human rights law 

the overseeing regulator, including the 
benefits of collaboration between different 
regulators with their relevant expertise, 
and the importance of resourcing the 
overseeing regulator to ensure a proactive 


There was support among stakeholders for the and strong regulatory approach.*”” 

use of regulatory sandboxes to ‘bake in’ human Stakeholders also noted the importance, in this 
rights protections in the design and development context, of protecting commercially-sensitive 
of emerging technologies.°”° information and intellectual property.°”° 
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(b) Conclusion: developing a regulatory 
sandbox for responsible Al 


AS Al is relatively new, it can be difficult to predict 
how it will operate in decision making and more 
broadly. Yet such prediction is crucial in creating 
the right regulatory settings. This problem can 
be exacerbated by a ‘move fast and break things’ 
ethos, which is common in the technology 
industry.” 


A result can be that Al-powered products and 
services are sometimes not being rigorously tested 
before being publicly released.” Inadequate 
testing can cause harm, including to human rights 
and public trust. High-profile examples include an 
Al-powered chatbot that made racist statements,°?" 
and a facial-recognition application that mislabelled 
some people with dark skin as gorillas.” 


Anticipatory regulation, however, is inherently 
difficult in the context of rapidly-evolving 
technology like Al. As a result, some laws that do 
not sufficiently address human rights risks, or are 
so restrictive that they disproportionately limit 
innovation. 


The Commission sees benefit in one or more 
Australian regulatory sandboxes, focused on 
assessing the human rights impact of Al-informed 
decision making. This could help to encourage 
such uses of Al to be tested more rigorously, and 
in developing effective regulation. 


Overseas experience supports a growing role for 
regulatory sandboxes to facilitate a co-regulatory 
approach that is of mutual benefit to the sandbox 
participant and the regulator. This would also be a 
way of Australia making good on its commitment 
to abide by the OECD Al Principles, which calls on 
governments to 


promote a policy environment that supports 

an agile transition from the research and 
development stage to the deployment and 
operation stage for trustworthy Al systems. To this 
effect, they should consider using experimentation 
to provide a controlled environment in which 

Al systems can be tested, and scaled-up, as 
appropriate.” 


To this end, the Commission recommends that an 
expert taskforce on Al advise on the development 
of one or more regulatory sandboxes focused on 

upholding human rights in the use of Al-informed 
decision making. 
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77 Human rights impact 
assessments 


RECOMMENDATION 15: The Australian 
Government should appoint an 
independent body, such as the Al Safety 
Commissioner (Recommendation 22), 

to develop a tool to assist private sector 


bodies undertake human rights impact 
assessments (HRIAs) in developing Al- 
informed decision-making systems. The 
Australian Government should maintain a 
public register of completed HRIAs. 


As discussed in Chapter 5, a human rights impact 
assessment (HRIA) is a tool to assess how a new 
product, service, law or policy will engage human 
rights. It also provides a framework for ensuring 
adequate rights protections. 


HRIAs are increasingly being used by government, 
the private sector and civil society organisations 
to measure the risk to human rights posed by 
their activities, ensure that measures are put in 
place to address human rights risks, and support 
the availability of remedies for any human rights 
infringements. 


Drawing on current research and the UN Guiding 
Principles on Business and Human Rights, the 
Danish Human Rights Commission recently 
published a toolkit for businesses seeking to 
conduct HRIAs, which emphasises: 


* using international human rights law as 
a basis and benchmark for the HRIA 


e applying a human rights approach, 
including principles such as participation, 
empowerment and transparency 


* accountability, recognising the rights owed 
to individuals and corresponding duties 
borne by governments, businesses and 
others.” 
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The Commission recommends that the Government 
develop an HRIA tool for use in Al-informed 
decision making. 


(a) Options for reform 


There was strong support, from the public and 
private sectors, for the Australian Government to 
develop an HRIA tool and associated guidance for 
Al-informed decision making.” Some stakeholders 
also observed that an HRIA will only protect 
human rights if situated within a strong regulatory 
framework.°*° 


Stakeholders also commented on a number of 
specific issues regarding HRIAs, including: 


* how the HRIA tool should be developed. 
Stakeholders generally submitted that there 
should be broad community consultation in 
developing an HRIA tool. Stakeholders also 
emphasised HRIAs should be developed by 
an independent body, such as an existing 
regulator or the Al Safety Commissioner 
(Recommendation 22), and be embedded 
with accountability measures.*?’ 


when an HRIA should be undertaken. 
Stakeholders were clear that the HRIA 
should be embedded throughout the Al 

life cycle, from the design of an Al product 
through to the monitoring and review of 

an Al-informed decision-making system in 
operation.**8 HRIAs should be undertaken 
by people with sufficient human rights 
expertise to ensure a meaningful process.” 


mandatory or voluntary. Some stakeholders 
supported making HRIAs mandatory, 
particularly where Al-informed decision 
making is being contemplated by 
government.™® Others observed that HRIAs 
may still have considerable influence even 
if not legally required, including where they 
are encouraged through measures such as 
government procurement rules.” 


* the consequences of a high risk finding. 
Stakeholders observed that where an HRIA 
highlighted a high risk of a human rights 
breach, especially before the relevant 
Al-informed decision-making system is in 
operation, this presents an opportunity 
to address the potential harm before any 
individual is negatively affected in reality.” 


how the HRIA should be applied to Al-informed 
decision-making systems developed overseas. 
Stakeholders recognised the difficulty in 
Australia regulating something, including 
an Al-informed decision-making system, 
that has been developed overseas.**? For 
this reason, there was a desire for Australia 
to work towards a harmonised approach 
internationally to undertaking HRIAs.>4 
Some stakeholders considered that, in 

any event, where such a system is to be 
used in Australia, it should be subject to an 
Australian HRIA process, and if it is non- 
compliant it should be prohibited.>*° 


(b) Conclusion 


The Commission considers that the use of HRIAs 
would help to identify and address human 
rights issues at the earliest stage of the design, 
development and deployment of Al-informed 
decision-making systems. 


The Australian Government's Al Action Plan 
recognises the need to assist corporations and 
other private sector bodies by ensuring the 
‘compliance burden of regulation is proportionate 
and specific to the risks associated with particular 
Al applications’.°“° Similarly, the Government's Al 
Ethics Principles note the importance of careful 
consideration of human rights risks.°*” 


The Australian Government could make good 

on this recognition by developing a tool to assist 
corporations and others to undertake HRIAs in 
this area. At this point, the Commission does not 
propose that HRIAs be mandatory for Al-informed 
decision making by non-government bodies. 


The effectiveness of any HRIA relies on it being a 
well-considered, rigorous process. In developing a 
tool for HRIAs in this area, it would be sensible to 
draw on other similar initiatives. These include: 


e the Canadian Government's algorithmic 
impact assessment process, which aims to 
identify and mitigate risks associated with 
automated decision making>® 


the Al Now Institute's algorithmic impact 

assessment accountability tool for public 
service use of algorithmic and automated 
decision making>*° 


Microsoft and Article One's HRIA tool, which 
can be used to examine human rights risks 
and opportunities related to Al.55° 


HRIAs are similar to other impact assessments, 
such as privacy impact assessments,°°' but are 
more comprehensive in addressing the full range 
of human rights engaged by Al-informed decision 
making. HRIAs could also help to provide tangible 
evidence of a company acting ethically. For 
instance, where a company completes an HRIA, it 
could help to show human rights compliance in a 
government procurement process. 
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78 Government procurement 


RECOMMENDATION 16: The Australian 
Government should adopt a human rights 
approach to procurement of products and 
services that use artificial intelligence. The 
Department of Finance, in consultation 
with the Digital Transformation Agency 


and other key decision makers and 
stakeholders, should amend current 
procurement law, policy and guidance to 
require that human rights are protected 
in the design and development of any Al- 
informed decision-making tool procured 
by the Australian Government. 


Procurement of goods and services by the 
Australian Government is regulated by a 
combination of laws, policies and guidance 
documents.*? The ‘core rule’ underpinning the 
Australian Government's procurement system is 
achieving ‘value for money’.**? Certain procurement 
processes by Government departments and 
agencies must comply with the Commonwealth 
Procurement Rules made under the Public 
Governance, Performance and Accountability Act 2013 
(Cth) 24 


The Digital Transformation Agency (DTA) has a 
central role in large-scale government procurement 
of information and communications technology 
(ICT). The DTA’s framework for ICT procurement 
includes best-practice principles to guide 
government agencies in sourcing ICT products and 
services. These principles include ‘encouraging 
competition’, and they urge agencies to be 
‘innovative and iterate often’, be ‘outcomes focused’ 
and ‘minimise cyber-security risks’. 


It is increasingly common to use government 
procurement processes as a lever to influence 
behaviour to achieve other policy outcomes. For 
example, Innovation and Science Australia identified 
government procurement to be a ‘strategic lever’ to 
stimulate and promote innovation.” Similarly, a UK 


parliamentary committee recommended ‘targeted 
procurement to provide a boost to Al development 
and deployment’ and ‘to capitalise on Al for the 
public good’.>*” 


The Australian Government, like other 
governments,’ has a significant level of committed 
investment in Al-powered products and services, 
and so it is natural that it should seek to influence 
the market positively in this area.°°? 


Moreover, the Australian Government generally 
develops Al-informed decision-making systems in 
partnership with the private sector. Consequently, 
it is vital that the Government procures Al-informed 
decision-making systems that are safe and protect 
human rights. The Commission recommends the 
Government review its procurement rules and 
policies to ensure they reflect a human rights 
approach in respect of products and services that 
use Al. 


(a) Options for improving government 
procurement 


There was broad support among stakeholders 

for government agencies to include human rights 
protections whenever they procure an Al-informed 
decision-making system.5® Stakeholders saw 
procurement rules as an important lever to secure 
transparent, explainable and accountable use of Al 
in decision making by government. 


Some stakeholders emphasised the often- 
sensitive nature of government decision making 
and potential impact on vulnerable people, which 
heightens the importance of ensuring that Al- 
informed decision making complies with human 
rights.>© 


Other stakeholders sought clarification on the 
meaning of human rights in this context.>° Given 
that Al-powered goods and services are often 
developed overseas, it could be difficult for 
Australia to impose higher or different standards 
compared with those adopted in larger markets.°“ 


The Al Now Institute suggested that these 
challenges could be addressed by including waivers 
regarding trade secrecy or other such commercial 
barriers to sharing information, referring to its 
Shadow Report to the New York City Automated 


Decision System Task Force.°° KPMG Australia 
submitted that a supplier from a high-risk 
jurisdiction should be required to demonstrate 
compliance with international human rights 
standards, rather than compliance only with local 
law.>©6 


(b) Conclusion 


Increasingly, governments are using Al to make 
decisions that have a momentous impact on the 
human rights of their citizens, including in relation 
to housing, health services, employment, social 
security and criminal justice. Those decisions can 
disproportionately affect people who are already 
vulnerable or disadvantaged. 


The Australian Government generally works 

with, and relies on, the private sector to develop 
Al-informed decision-making systems. It is 

well recognised, therefore, that government 
procurement should focus on ensuring these 
systems are safe and comply with human rights.°° 


The UK Government's Artificial Intelligence 
Office, for example, published guidelines for Al 
procurement in 2020. The ten guidelines focus 
on ensuring the specific risks associated with Al 
technology are identified and managed early in 


the procurement phase, and they recommend 
explainability and interpretability of algorithms be 
included as design criteria.” The UK Government 
partnered with the World Economic Forum and 
industry bodies to produce a procurement toolkit to 
guide responsible—and human rights compliant— 
procurement of Al by the public sector (see 

Box 7).°° 


Civil society organisations have also recognised the 
potential of government procurement to incentivise 
the development of human rights compliant Al for 
use in decision making. Access Now, for example, 
has recommended open procurement standards be 
followed by the public sector, including ‘publication 
of the purpose of the system, goals, parameters, 
and other information to facilitate public 
understanding’, including providing for a period 

for public consultation, particularly with affected 
groups.°”° 


Similarly, the Al Now Institute has developed 
detailed questions to support transparent and 
accountable public sector procurement of Al- 
informed decision-making systems, such as 
requiring vendors to provide detailed information 
regarding training and input data for the Al tool, 
including plain language descriptions regarding 
how the system makes determinations.” 


Box 7: World Economic Forum Al Government Procurement Guidelines 


The World Economic Forum has proposed ‘actionable procurement guidelines’ for government 
procurement of Al-powered products and services, to ‘enable good decision-making that can also 
be evaluated’.°”? Acknowledging that public procurement has been shown to deliver strategic 
outcomes in areas such as human rights, it notes that poor procurement decisions in respect of Al 
‘can limit accountability, undermine social values, entrench the market power of large businesses, 
decrease public trust and ultimately slow digital transformation in the public sector.” 


The toolkit offers guidance on matters such as using Al risk and impact assessments—including 
HRIAs—to inform the procurement process and ensure mitigation strategies are adopted.°” 
Other key considerations include: the availability of technical information and any governance 
considerations of obtaining relevant data; whether limitations on using personal data may render 
the end product inappropriate, unreliable or misleading; feedback obtained from a diverse and 
multidisciplinary team, and requiring tender information to include information about the design 


team of the Al system.°”> 
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The Australian Government should adopt a human 
rights approach to government procurement of 
important products and services that use Al. This 
will ensure that the Government continues to 
meet its human rights obligations through any 
outsourcing or contracting arrangement. This 
process should consider: 


* amending the DTA’s Digital Sourcing 
Framework for ICT procurement to include 
a specific reference to human rights 
protection 


a requirement for any vendor of an 
Al-informed decision-making system 
to complete a human rights impact 
assessment 


guidance to support government decision 
makers to assess whether an Al-informed 
decision-making system will support 
compliance with the legal measures 
guaranteeing a right to a remedy°”® 


compliance with relevant Australian and 
international standards. 


7.9 Monitoring and evaluation: 
human oversight and intervention 


RECOMMENDATION 17: The Australian 
Government should engage an expert 
body, such as the Al Safety Commissioner 
(Recommendation 22), to issue guidance 
to the private sector on good practice 
regarding human review, oversight and 


monitoring of Al-informed decision- 
making systems. This body should also 
advise the Government on ways to 
incentivise such good practice through the 
use of voluntary standards, certification 
schemes and government procurement 
rules. 


There is an important role for people in overseeing, 

monitoring and intervening in Al-informed decision 
making. Human involvement is especially important 
to: 


* review individual decisions, especially to 
correct for errors at the individual level 


* oversee the operation of an Al-informed 
decision-making system to ensure the 
system is operating effectively as a whole. 


The Commission recommends that the Australian 
Government provide guidance for the private sector 
on good practice for human review of Al-informed 
decisions. There may be a case for making human 
review mandatory for the private sector. However, 
in the first instance, the Commission recommends 
that the Government focus on methods to 
incentivise human review through the use of 
voluntary standards, certification and government 
procurement rules. 


(a) Options for review of Al-informed 
decisions 


Adopting an Al-informed decision-making system 
involves choosing the extent to which humans will 
be involved in the decision-making process. It is 
extremely rare to adopt a wholly automated or 
‘unsupervised’ system, with no human involvement 
or oversight. 


The level and quality of human involvement in 
an Al-informed decision-making system will be 
determined by the answers to questions such as 
the following: 


* To what extent is the decision-making 
process automated or unsupervised? 
In other words, is the decision made 
wholly using Al, or is Al used to provide a 
recommendation or other information that 
a human can consider in making a decision? 


What kind of human oversight is provided 
for? Can a human overturn and replace a 
decision made using Al? 


What information about the operation 

of Al is provided to the human fulfilling 
the oversight function? Is the human 
appropriately qualified to understand this 
information? 


* To what extent do humans monitor the 
operation of the system as a whole, making 
any adjustments necessary to improve its 
operation? 


The European Commission's Ethics Guidelines for 
Trustworthy Al summarises three levels of human 
involvement in an Al-informed decision-making 
system: 


e ‘human-in-the-loop’, which is the ‘capability 
for human intervention in every decision 
cycle of the system’- although the 
Guidelines also acknowledge that ‘in many 
cases [this] is neither possible nor desirable’ 


‘human-on-the-loop’, where humans are 
involved in designing the system and 
monitoring its operation 


‘numan-in-command’, which is the human 
‘capability to oversee the overall activity 

of the Al system (including its broader 
economic, societal, legal and ethical 
impact) and the ability to decide when and 
how to use the system in any particular 
situation’ including deciding not to use Al, 
establishing levels of human discretion or 
giving human decision makers the ability to 
override a decision.°”” 


Many stakeholders expressed deep concern about 
the phenomenon of Al-informed decision making 
that is wholly unsupervised or has inadequate 
supervision by humans. They argued that human 
oversight is essential at all critical points in the 
design, development, use and monitoring of Al- 
informed decision making.°” 


There was broad concern—among civil society, 
government and academic stakeholders—that 
there is often inadequate human oversight for Al- 
informed decision making by the private sector, 
especially given such decisions can significantly 
affect individuals’ human rights.°”? 


Some stakeholders saw a right of review by 

a human decision maker as being the most 
appropriate way of involving humans in Al-informed 
decision making.*®° There was also support for 
oversight of the overall operation of Al-informed 
decision-making systems, such as by way of audit, 
to ensure ongoing accountability for the system's 
outputs.%* 


However, there was opposition to the idea of 
introducing a general legal requirement on the 
private sector to adopt a particular form of human 
oversight for Al-informed decision making. It 
was observed that human involvement does 

not necessarily improve the quality of decision 
making.°® For example, human involvement can 
introduce human bias or prejudice.” Moreover, 
for some tasks, Al-powered applications already 
outperform humans, making it difficult to design 
oversight in a way that improves the decision- 
making process.” 


(b) Conclusion: improving review and 
oversight of private sector use of Al 


(i) Reviewing individual decisions 


Where government makes administrative decisions 
using Al, affected people should be able to seek 
independent merits review and judicial review.°® 


Where a private sector body makes an Al-informed 
decision, it is good practice for the body to allow 
an affected person to have the decision reviewed 
by a human with appropriate authority, skills and 
information. The purpose of this review would 

be to identify and correct any errors that have 
arisen through the use of Al in the decision-making 
process. 


The Commission acknowledges that providing for 
such review can be difficult. Often there will be a 
need for a significant level of technical knowledge 
regarding the form of AI.586 Sometimes expertise 
from outside the organisation that made the 
decision will be needed.**” 


In view of such challenges, the Commission 
recommends that the Australian Government 
provide guidance for the private sector on effective 
human review of Al-informed decisions. Ideally, this 
should be developed by the Al Safety Commissioner 
(Recommendation 22). The Commission also 
recommends that options be considered for 
incentivising human review through the use of 
voluntary standards, certification and government 
procurement rules. 
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7. Encouraging better Al-informed decision making 


Some overseas jurisdictions have gone further, 
requiring a form of human review—at least for 
some types of Al-informed decision made by non- 
government bodies. In particular, Article 22 of the 
GDPR provides for ‘human intervention’ to enable 
an individual affected by an automated decision to 
express their view and contest the decision. 


The Commission does not presently recommend 
that private sector bodies be /egally required to 
make provision for human review. While this view 
may change, the Commission is not currently 
convinced that mandating human review by private 
sector bodies would be desirable. The reasons for 
this view include: 


e Adifferent system of legal accountability 
applies to decisions made by the private 
sector, as compared to administrative 
decisions by or on behalf of government. 
There is no equivalent to the administrative 
law system of review for private sector 
decision making. 


There are other legal protections, which 

can be adjudicated by courts and tribunals, 
that apply to private sector decision 

making. For example, where a company is 
responsible for an Al-informed decision that 
involves unlawful discrimination, this can be 
challenged in courts and tribunals. 
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* Given the breadth of Al-informed decision 
making, the almost infinite potential for 
it to expand further, and the fact that 
some decisions cannot currently be made 
more accurately by humans without the 
assistance of Al, it would be very difficult 
to legislate a specific form of review that 
would be suitable for all types of Al- 
informed decision making. 


(ii) System-level oversight 


The Government and private sector also should 
consider the need for human oversight of the 
operation of Al-informed decision-making systems. 
The aim of this form of human oversight is to 
ensure the system is operating effectively as a 
whole. 


The Commission considers that such oversight 
should occur at all stages of the Al lifecycle: in the 
design and development of Al-informed decision- 
making systems; when these systems are being 
tested in a controlled environment; and at regular 
intervals after they have begun operating ‘in the 
real world’. 


As with human review of individual Al-informed 
decisions, the Commission recommends that the 
Australian Government provide guidance in respect 
of system-wide oversight and monitoring of these 
systems, and consideration be given to incentivising 
such activity through standards, certification and 
government procurement rules. 


8. Al, equality and non- 
discrimination 


8.1 Summary 


Artificial intelligence can enable good, data-driven decision making. The use of Al can 
allow large amounts of relevant information to be considered, sifting out anything that is 
irrelevant to the decision-making process. 


However, there is a growing awareness that the use of Al does not always achieve this 
objective. It can sometimes result in decisions that are unfair or even discriminatory. This 
problem is often referred to as ‘algorithmic bias’. 


Unfairness and discrimination can arise in all forms of decision making, not just 
Al-informed decisions. The problem sometimes stems from conscious, as well as 
subconscious or unconscious, bias and prejudice. However, the use of Al can have the 
effect of obscuring and entrenching unfairness or even unlawful discrimination.” 


This chapter sets out some of the key issues relating to algorithmic bias, and its 
connection to unlawful discrimination. It recommends greater guidance for government 
and non-government bodies in complying with anti-discrimination law in the context of 
Al-informed decision making. 


8.2 Al and the risk of discrimination 


Under international law, unlawful discrimination occurs when a person is treated less 
favourably than another person or group because of their race, colour, sex, language, 
religion, political or other opinion, national or social origin, property, birth or other 
status.°8° 


This right has been largely incorporated into Australia’s federal, state and territory 
laws. Federal anti-discrimination laws prohibit discrimination on the basis of protected 
attributes that include an individual's: 


* age 
e disability 


race, including colour, national or ethnic origin or immigrant status 


* sex, pregnancy, marital or relationship status, family responsibilities or 
breastfeeding 


sexual orientation, gender identity or intersex status.°°° 


Some state and territory laws offer more extensive anti-discrimination protection in their 
respective jurisdictions, as compared with federal law.” 
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8. Al, equality and non-discrimination 


Under federal law, it is unlawful to discriminate 
against a person on the basis of a protected attribute, 
in providing or refusing to provide goods, services or 
facilities.°°* Discrimination can be direct or indirect. 
‘Direct discrimination’ is where a person is treated 
differently from others. By contrast, ‘indirect 
discrimination’ occurs when an unreasonable rule 
or policy applies to everyone but has the effect of 
disadvantaging some people because of a personal 
characteristic they share. 


Indirect discrimination can arise where apparently 
innocuous information is, in reality, a ‘proxy’ or 
indicator for a protected attribute. For example, 

if large numbers of people of the same ethnic 
origin happen to live in a particular suburb or 
neighbourhood, the postcode where an individual 
lives can become a proxy or likely indication of the 
individual's ethnic origin." If a decision is made by 
reference to that proxy, and the decision unfairly 
disadvantages members of that ethnic group, it 
could lead to indirect discrimination. 
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(a) Algorithmic bias 


While ‘algorithmic bias’ is not a term of art, it is 
usually understood to refer to the situation where 
Al is used to produce outputs that treat one group 
less favourably than another, without justification. 
Algorithmic bias can include statistical bias—long 
familiar to computer scientists, data analysts 

and statisticians—as well as concepts of fairness, 
equality and discrimination. It can arise through 
problems with the data being used by the Al- 
powered system or tool, or because of problems 
with the system or tool itself. 


The use of Al can assist in identifying and 
addressing bias or prejudice that can be present 
in human decision making,” but it can also 
perpetuate or entrench such problems. In a recent 
review of algorithmic decision making in the 
recruitment, financial services, policing and local 
government sectors, the UK Government's Centre 
for Data Ethics and Innovation concluded: 


New forms of decision-making have surfaced 
numerous examples where algorithms have 
entrenched or amplified historic biases; or even 
created new forms of bias or unfairness. Active 
steps to anticipate risks and measure outcomes 
are required to avoid this.°° 


Examples of the problem of algorithmic bias, 

which in some situations can involve unlawful 
discrimination, are emerging in decision making in 
the criminal justice system, advertising, recruitment, 
healthcare, policing and elsewhere. 


Where an algorithm is expressly designed to 
exclude a particular group, or where it gives extra 
weight to a protected attribute such as race, age, 
or gender, it is likely to disadvantage people by 
reference to those protected attributes. In these 
situations, discrimination may be easy to identify. 


However, unfairness or discrimination also can be 
difficult to detect and address. Much will depend 
on the data used to train an Al-informed decision- 
making system.°% Some refer to data science’s 
‘garbage in, garbage out’ problem, where a ‘flawed’ 
data set is used to produce decisions that are 
unreliable, unfair or discriminatory. 


Such flaws can arise, and lead to algorithmic bias, 
for a variety of reasons. Examples include where 
the Al-informed decision-making system is designed 
in a way that: 


e gives undue weight to a particular data set 


e relies on a data set that is incomplete, out 
of date or incorrect, or 


e uses a data set that is affected by selection 
bias—that is, where the data set is not 
representative of a population so may 
ultimately favour one group over another.°?” 


There has been growing attention to the problem 
that arises where an Al-informed decision-making 
system is ‘trained’ on historical data that is affected 
by prejudice or unlawful discrimination. 


For instance, imagine Al is used to make home 
loan decisions. If the Al-informed decision-making 
system is trained on many years of human 
decisions that were prejudiced against female loan 
applicants—in other words, if the training data 
contains a historical bias—the system can replicate 
or even reinforce this bias in its outputs. 


This historical bias might be ‘hidden’ in the training 
data, in the sense that it is difficult to discern 

the unfair disadvantage. Yet the Al-informed 
decision-making system will continue to apply this 
disadvantage to female loan applicants, even if 
there is no longer any underlying prejudice or other 
improper motivation in the design of the system.°% 


An oft-cited example is a recruitment tool that 
favoured male over female candidates. The 
algorithm was trained to identify patterns in job 
applications received by the company over a 10- 
year period. As most of the job applicants were 
male, the system ‘learned’ that male applicants 
were preferable, and generated recommendations 
for the future workforce accordingly.°” 


Similarly, profiling individuals through data 
mining in order to draw inferences about 

their behaviour carries risks of unfair and 
discriminatory treatment.® This, too, can lead to 
unlawful discrimination. The UN Committee on 
the Elimination of Racial Discrimination recently 
completed a three-year study on the use of Al 

in a policing context, identifying a greater risk of 
racial profiling arising from the use of certain Al 
methods. 
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8. Al, equality and non-discrimination 


8.3 Anti-discrimination law reform 
or guidance 


RECOMMENDATION 18: The Australian 
Government should resource the 
Australian Human Rights Commission to 


produce guidelines for government and 
non-government bodies on complying 
with federal anti-discrimination laws in 
the use of Al-informed decision making. 


In principle, it is unlawful to discriminate in Al- 
informed decision making, just as it is in more 
conventional decision making. However, novel 
problems can arise in applying anti-discrimination 
law to Al-informed decision making. 


For example, determining whether an individual 
has been treated differently based on a protected 
attribute can be difficult where there is a 
combination of variables entered into a machine- 
learning algorithm. If reasons or an explanation 
are not produced for an Al-informed decision, this 
can make it even more difficult, if not impossible, 
to determine whether unlawful discrimination has 
occurred. 


Stakeholders requested the Commission or 
another expert body issue guidance on avoiding 
discrimination in Al-informed decision making.® 
Depending on how it is used, Al can either reduce 
or increase the risk of unfairness and unlawful 
discrimination. Consequently, this chapter 
recommends that the Commission be resourced to 
produce detailed guidance in this area. 


The Australian Government's AI Ethics Principles 
state that Al systems ‘should not involve or result 
in unfair discrimination’. The Principles recognise 
a responsibility on those engaging in Al-informed 
decisions to take measures that ensure these 
‘decisions are compliant with anti-discrimination 
laws’.603 


Similarly, the Commission endorses the UK 
Government's Centre for Data, Ethics and 
Innovation observation that 


the current focus should be on clarifying how 
existing legislation applies to algorithmic decision- 
making, ensuring that organisations know how 

to comply in an algorithmic context, alongside 
effective enforcement of these laws to algorithmic 
decision-making... [T]he application of current 
legislation must be clear and enforced accordingly 
to ensure bad practice is reduced as much as 
possible.“ 


The Commission partnered with a number of 
leading expert organisations to publish a technical 
paper for Australian businesses on understanding 
and addressing algorithmic bias in how they design 
Al-informed decision-making systems (see Box 8).°° 


It is clear, however, that more guidance is needed. 
In the UK, it was noted that national human 
rights institutions are well placed to issue this 
sort of guidance.° As Australia’s national human 
rights institution, the Commission is the principal 
authority on Australia’s anti-discrimination laws, 
and has extensive experience in investigating 
and conciliating complaints of discrimination 

and human rights breaches. Under s 11(1)(n) of 
the Australian Human Rights Commission Act 1986 
(Cth), the Commission has the power to produce 
guidelines on relevant laws.°” 


To this end, there would be value in the 
Commission producing detailed guidance on 

the application of anti-discrimination law to Al- 
informed decision making for Australian public and 
private sector bodies. The Australian Government 
should provide the Commission with the necessary 
resources to undertake this role. 


A number of the recommendations regarding the 
accountability of Al-informed decision making, set 
out in Chapters 5 and 6, would help to address 
the problem of algorithmic bias. In addition, there 
may be a need for more targeted reform to anti- 
discrimination law, to address issues such as: 


e how reliance on a particular factor, or 
combination of factors, may act as a proxy 
for a protected attribute, and how this 
applies to the legal concepts of direct and 
indirect discrimination 


e whether particular techniques associated 
with Al, such as predictive machine learning 
algorithms, could lead to new forms of 
discrimination through the drawing of 
connections by reference to a multitude of 
data points.°% 


There is a growing body of research that is 

leading to the development of measures that aim 
to address the problem of algorithmic bias. In 
assessing the need for reform, it would be useful to 
consider such measures as: 


e ‘preprocessing methods’, referring to 
strategies to address problems in the 
training data that can result in algorithmic 
bias°°? 


e ‘in-processing’ techniques, involving 
modifying the learning algorithm 
* ‘post-processing’ methods, involving 


auditing algorithmic outcomes to identify 
and resolve discrimination patterns.®"° 


COMPEAANCE ° 
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Box 8: Technical Paper: Addressing 
the problem of algorithmic bias 


In November 2020, the Commission 
published a Technical Paper, Using 
artificial intelligence to make decisions: 
Addressing the problem of algorithmic 
bias. The Paper was the product of a 
partnership between the Commission, 
Gradient Institute, Consumer Policy 
Research Centre (CPRC), CHOICE and 
CSIRO's Data61. 


Using a synthetic data set, the Technical 
Paper tests how algorithmic bias can 
arise, using a hypothetical simulation: an 
electricity retailer using an Al-powered 
tool to decide how to offer its products to 
customers, and on what terms. 


The simulation identified five forms of 
algorithmic bias that may arise due to 
problems attributed to the data set, the 
use of Al itself, societal inequality, or a 
combination of these sources. 


The Paper investigates if algorithmic bias 
would be likely to arise in each scenario, 
the nature of any bias, and provides 
guidance regarding how these problems 
might be addressed. Specifically, it shows 
how these problems can be addressed by 
businesses acquiring more appropriate 
data, pre-processing the data, increasing 
the model complexity, modifying the Al 
system and changing the target variable. 


The Paper, the first of its kind in 

Australia, highlights the importance 

of multidisciplinary, multi-stakeholder 
cooperation to produce practical guidance 
for businesses wishing to use Al in a way 
that is responsible and complies with 
human rights. 
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9. Biometric surveillance, facial 
recognition and privacy 


9.1 Summary 


Personal information is the ‘fuel’ that powers Al. It is well recognised that Al poses 
particular risks to individuals’ control of their own personal information and the right to 
privacy. Seemingly innocuous personal data can be used, especially in an Al-powered 
system, to gain insights about an individual, including on sensitive matters.°" 


The use of Al in biometric technology, and especially some forms of facial recognition, 
has prompted growing public and expert concern.°? 


Biometric technology involves using one or more of an individual's physical or biological 
characteristics to identify that person, or to discern other things about that person. 
This field is not new: fingerprints, for example, have been used for well over a century 
to identify people. However, biometric technology has been supercharged by Al, and 
specifically by new capabilities to analyse large sources of data. 


Currently, biometric technology can identify people by reference to their face, eyes, 
genetic or DNA material, and myriad other physical features. The technology can be used 
to verify an individual's identity or to identify someone from a larger group. There are 
also attempts to use the technology to gain other insights about people, such as their 
mood or personality. 


This necessarily affects individual privacy, and can fuel harmful surveillance. In addition, 
certain biometric technologies are prone to high error rates, especially for particular 
racial and other groups. Where these biometric technologies are used in high-stakes 
decision making, like policing, errors in identification can increase the risk of injustice 
and other human rights infringement. 


The specific type of biometric technology being used, and how it is used, will affect 
the degree of human rights risk. Some, but not all, uses of biometric technology carry 
significant risks. This Project has focused primarily on high-risk uses of biometric 
technology, and especially facial recognition, in contexts where the consequences 

of error can be grave. 


As discussed below, the Commission recommends privacy law reform to protect against 
the most serious harms associated with biometric technology. Australian law should 
provide stronger, clearer and more targeted human rights protections regarding the 
development and use of biometric technologies, including facial recognition. Until these 
protections are in place, the Commission recommends a moratorium on the use of 
biometric technologies, including facial recognition, in high-risk areas. 
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9. Biometric surveillance, facial recognition and privacy 


9.2 Development and use of 
biometric technology 


Biometric identification involves a process of 
identifying people by reference to their biological 
information. At least in theory, almost any biometric 
information could be used to identify individuals— 
including facial features, fingerprint, gait and even 
body odour.®"4 In scientific terms, those categories 
are limited only by two criteria: whether the 
biometric information in question can be accurately 
measured, and whether this information is truly 
unique to each individual. 


Biometric identification has taken place using 
fingerprints since the 19" century. Advances in 
technology, especially in recent decades, have 
enabled new forms of biometric identification, 
often on a scale and speed that were previously 
unachievable. The physical characteristics that 
can be used to identify people are increasing; and 
biometric technology is being used in an ever- 
increasing range of ways and domains. 


Biometric identification has become common as 


an alternative to a password or key, to enable 
access to products and services. While such uses 
are largely uncontentious, the potential for other 
uses is almost limitless. Some uses are more 
controversial, such as the use of facial recognition 
in policing and law enforcement. 


Biometric identification starts with the collection 
and storage of biometric data from individuals. 
Australian privacy law, like that of many other 
jurisdictions, treats biometric information as 
personal information. As a general rule, biometric 
data can be used only for the purpose for which it 
was originally collected. For example, if a company 
collects an individual’s fingerprint for the purpose 
of enabling the individual to use their finger to 
unlock their smartphone, the company would not 
be permitted to use the individual's fingerprint for 
an unrelated purpose, such as to build a database 
of people whose information could then be sold to 
a third party for marketing purposes. 


However, some uses of personal information do 
not require the consent of the individual, and 
even where consent is required, this can provide 
inadequate protection.®'® Where, for example, a 
company obtains consent from an individual to use 
their biometric data for a vague or broad range of 
purposes, this may allow the company to use the 
data in ways that the individual might not have 
initially contemplated. The individual might not in 
fact be comfortable with some of those later uses, 
and indeed some might be harmful. 


The Discussion Paper proposed two reforms that 
would have a particular impact in this area: 


* the creation of a statutory cause of action 
for serious invasion of privacy 


* the introduction of a limited moratorium 
on the use of facial recognition technology 
in high-risk decision making, at least until 
a legal framework has been established to 
safeguard human rights.°"® 


Concern about high-risk forms of facial recognition, 
and other biometric technologies, has continued 

to grow—especially in connection with law 
enforcement, access to government services and 
surveillance by private companies.°"” 


Some governments have passed or proposed laws 
regulating the use of facial recognition, and some 
courts have applied human rights and other laws to 
restrict the use of facial recognition.*'® Some leading 
companies have also imposed their own constraints 
on how they will use biometric technology such 

as facial recognition.°'? Nevertheless, in the past 
year, there has been continued growth in the 
development and use of this kind of technology, 
including to address issues that have arisen in the 
context of the COVID-19 pandemic.°° 


9,3 Facial recognition technology 
and human rights 


Facial recognition technology is used by 
governments, corporations and others in Australia 
and elsewhere.' There are two principal forms of 
facial recognition technology: ‘one-to-one’ and ‘one- 
to-many’. 


One-to-one facial recognition involves a computer 
checking whether a single headshot photograph 
matches a different headshot of the same person. 
It can be used to verify whether an individual is 
who they claim to be, performing a similar task 

to a password or key. Many of us use one-to-one 
facial recognition to unlock smartphones and other 
electronic devices. 


One-to-many facial recognition also seeks to 

match a single headshot with a different stored 
headshot of the same individual. The difference, 

as compared with one-to-one facial recognition, is 
that the matching headshot is located somewhere 
in a larger database or store of headshots of other 
people. Sometimes that database can be vast. Thus, 
from a technical perspective, identifying someone 
using one-to-many technology is considerably more 
difficult, and more prone to error.°2 
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While one-to-one facial recognition is often used to 
determine whether an individual is who they claim 
to be, one-to-many facial recognition is commonly 
used to answer a more difficult question: who is 
this person? This means that one-to-many facial 
recognition can be put to a wider range of uses and, 
as discussed below, presents greater human rights 
risks. Those risks relate both to the consequences 
of error in Al-informed decision making, as well as 
broader risks where this form of facial recognition 
is used to cause harm, or in ways that undermine 
the right to privacy. 


(a) Human rights risks associated with facial 
recognition 


Like most technologies, facial recognition is neither 
inherently good nor bad. 


Some stakeholders emphasised how facial 
recognition could be used to benefit individuals 
and the community at large. For example, the 
technology has been used to detect and prosecute 
child exploitation, to address human trafficking, 

to investigate fraud, and to identify missing 
persons, including after the Australian bushfires in 
2019-20.°3 


However, amid a growing recognition of the risks 
associated with biometric technology in recent 
decades,°** most stakeholders focused primarily on 
how facial recognition could be used to limit human 
rights. Concern focused on three risks: 


e the contribution of facial recognition to the 
growth in surveillance 


* the use of data derived from facial 
recognition to engage in profiling 


e the risk that errors connected to facial 
recognition disproportionately affect certain 
groups 


In addition, the use of facial recognition in the 
private sector raises distinct concerns as there may 
be a lower degree of accountability and fewer legal 
protections. 


(i) Surveillance 


Many stakeholders were concerned about the 
growing risk of widescale or mass surveillance.°° 
The inevitable reduction of personal privacy, and 
the threat of closer scrutiny by police and other 
government agencies, can inhibit participation 

in lawful democratic processes such as protests 
and some meetings. This engages the rights to 
freedom of association and assembly; freedom of 
expression and opinion; and, potentially, the right 
to be free from unlawful and arbitrary arrest. 


Some have argued that the widespread use of 
facial recognition, especially in public places, 
disproportionately limits these human rights. For 
example, Access Now Stated that public surveillance 
is ‘neither necessary nor proportionate to the goal 
of public safety or crime prevention, and therefore 
violate[s] the right to privacy.’ 


In 2018, the Council of Europe Rapporteur 

on human rights and Al, Professor Karen 
Yeung, concluded that the cumulative impact 
of mass surveillance poses a serious risk to 
the ‘fundamental social conditions’ that make 
protection of individual rights possible. When 
combined with data-driven technologies, which 
‘enable fairly innocuous and mundane data to 
be merged and mined in ways that may reveal 
highly personal characteristics’, these forms of 
surveillance 


can be very powerful tools in the hands of 
governmental regimes, whether liberal or 
repressive, and therefore generate acute threats to 
the exercise of all human rights and fundamental 
freedoms.% 


Lawmakers overseas have expressed concern 
about the prospect of facial recognition being used 
to identify and arrest people engaging in lawful 
protest, such as in the 2020 Black Lives Matter 
protests across the US.°9 


(ii) Profiling 

As personal data becomes more widely collected, 
stored and analysed, a small piece of personal 
information—apparently inconsequential on 

its own—can be combined with other personal 
information to enable detailed insights about the 


person in question. This has been dubbed the 
‘mosaic effect’.©° 


Personal data, derived from increasingly 
widespread facial recognition tools, can be 
combined with other data sources. As with a 
mosaic or puzzle, the aggregated data can then be 
used to create a more detailed picture: inferences 
about the individual may be drawn, and these can 
sometimes be shared with third parties without any 
meaningful consent from the affected individual.®"' 
The inclusion of biometric information can allow 
sensitive personal information to be extracted or 
inferred, including in relation to the individual's age, 
race, sex and health. 


Such information and inferences can be used 

in ‘profiling’—where intrusive action is taken by 
reference to people's characteristics. An example 
of this kind of profiling, which may result in 
people of a particular racial or ethnic group being 
disproportionately subjected to police identity and 
other checks, has emerged in China.°? 


Since November 2017, the UN Committee on the 
Elimination of Racial Discrimination (CERD) has 
been consulting widely on this problem. That 
process culminated in the publication, on 24 
November 2020, of a new general recommendation 
on the /nternational Convention on the Elimination 

of All Forms of Racial Discrimination. General 
Recommendation 36 observed: 


The increasing use of facial recognition and 
surveillance technologies to track and control 
specific demographics raises concerns with 
respect to many human rights, including the right 
to privacy, freedom of peaceful assembly and 
association; freedom of expression and freedom 
of movement. It is designed to automatically 
identify individuals based on their facial geometry, 
potentially profiling people based on grounds of 
discrimination such as race, colour, national or 
ethnic origin or gender.°? 


(iii) Errors and the risk of discrimination 


Currently, facial recognition technology is generally 
less accurate when identifying women, or people 
from minority racial groups, as compared with 
other people.®4 To this end, CERD recently 
concluded: 


[I]t has been demonstrated that the accuracy 
of facial recognition technology may differ 
depending on colour, ethnicity or gender 

of the persons assessed, which may lead to 
discrimination.®° 


The UK Centre for Data Ethics and Innovation 
recently summarised research on levels of 
inaccuracy of facial recognition systems used in that 
country.®° The Centre concluded that 


the distribution of false positives and incorrect 
interventions will depend on the demographic 
make-up of the watch list, as well as that of the 
people scanned, even where the underlying 
algorithm has no in-built bias.°” 
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Concerns were also raised by stakeholders in 
relation to reported lower accuracy rates for non- 
white faces.°° The Castan Centre for Human Rights 
Law observed that this means ‘certain racial groups 
will likely face more frequent misidentification and 
be subject to increased police scrutiny.’°?° Access 
Now was especially concerned about the impact 
on Aboriginal and Torres Strait Islander peoples, 
with these groups being more likely to experience 
errors, including identification as a suspect, in the 
prosecution of criminal offences.©? 


Errors in facial recognition can be more likely 
among groups such as Aboriginal and Torres Strait 
Islander peoples because the datasets typically 
used to train facial recognition tools tend to have 

a very low number of headshots of people from 
these groups. One way of addressing that problem 
would be to increase the representation of people 
from those groups in these training datasets, 

but that carries its own human rights risks.%! 

For example, where people from these groups 
experience harm from the use of facial recognition, 
it is understandable that they would be reluctant to 
participate in the development of these systems. 


(iv) Use of facial recognition in the private 
sector 


The use of facial recognition technology is not 
limited to the police or the public sector more 
broadly. Increasingly, these tools are being used 

in workplaces, schools, shopping centres and 
residential areas to identify members of the 

public and monitor behaviour.® In those settings, 
there are often fewer accountability and other 
protections against misuse and harm, and public 
concern about the use of this technology by private 
companies remains high. 


The OAIC’s public polling, for example, provided 
evidence that most Australians are uncomfortable 
with the collection by retail stores of customers’ 
biometric information, including via facial scans.6# 


9.4 Regulation of biometric 
technology 


RECOMMENDATION 19: Australia’s 
federal, state and territory governments 
should introduce legislation that regulates 
the use of facial recognition and other 
biometric technology. The legislation 
should: 


(a) expressly protect human rights 


(b) apply to the use of this technology 
in decision making that has a legal, 
or similarly significant, effect for 


individuals, or where there is a high 
risk to human rights, such as in 
policing and law enforcement 


(c) be developed through in-depth 
consultation with the community, 
industry and expert bodies such 
as the Australian Human Rights 
Commission and the Office 

of the Australian Information 
Commissioner. 


RECOMMENDATION 20: Until 

the legislation recommended in 
Recommendation 19 comes into 

effect, Australia’s federal, state and 
territory governments should introduce 
a moratorium on the use of facial 


recognition and other biometric 
technology in decision making that has 

a legal, or similarly significant, effect for 
individuals, or where there is a high risk to 
human rights, such as in policing and law 
enforcement. 


There are many positive uses of facial recognition 
and other biometric identification technology. 
However, the risks of harm, including to human 
rights, are real. Regulation should be carefully 
targeted to address harmful use of biometric 
technologies. 


The risks associated with facial recognition are 
generally highest where this technology is used in 
decision making that affects an individual's legal 

or similarly significant rights. This is most obvious 
when the technology fails. For example, if an error 
in the facial recognition tool on a smartphone 
causes a delay in an individual ‘unlocking’ their 
device, generally this would present little more 
than an annoyance. However, if a person is wrongly 
accused of a crime on the basis of an error in police 
use of facial recognition, the risk of harm is far 
greater. 


The primary way of addressing the latter problem 
is for the law to require all such policing decisions 
to be made accurately, fairly and without 
discrimination—regardless of whether they are 
made using facial recognition. However, where 
there are concerns about the reliability of the 
underlying technology—as is the case with one-to- 
many facial recognition, especially in identifying 
people with, for example, darker skin—this can 
suggest a need for further regulation that is 
directed towards the technology itself. 


Many stakeholders similarly urged that the 
regulatory approach to facial recognition should 
focus on risk.°4 


The Commission recommends that all Australian 
governments work cooperatively to introduce 
legislation to regulate the use of facial recognition 
and other biometric technologies. Until appropriate 
legislation is in effect, the Commission recommends 
there be a moratorium on the use of facial 
recognition and other biometric technology in Al- 
informed decision making. 


This moratorium would not apply to a// uses of 
facial and biometric technology. It would apply only 
to uses of such technology to make decisions that 
affect legal or similarly significant rights, unless and 
until legislation is introduced with effective human 
rights safeguards. 


(a) Use of facial recognition technology 
in policing and law enforcement 


Police and law enforcement are increasingly using 
one-to-many facial recognition to identify criminal 
suspects, detect crimes, and help find missing 
persons. 


While these objectives are legitimate and can 
promote community safety, there are significant 
human rights risks associated with the use of facial 
recognition for these ends. In particular, error can 
result in suspects, victims and witnesses being 
wrongly identified. This can limit human rights 
including the right to equality or non-discrimination, 
the rights to equality before the law, personal 
security and liberty, the right to a fair public hearing 
and the right to procedural fairness and due 
process, including the presumption of innocence. 
More broadly, the overuse of facial recognition 

by police can disproportionately limit the right to 
privacy and can contribute to mass surveillance. 


There is strong and growing public concern about 
the use of facial recognition technology in policing 
and law enforcement and the problems it may 
cause, including in relation to human rights and 
privacy. 


Courts may increasingly be required to consider 
issues raised by facial recognition technology. An 
early landmark case in this area was the Bridges 
Case, considered in Case Study 2. 
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Case Study 2: 


The Bridges Case 


The Court of Appeal of England and Wales (the Court) considered a challenge to the use of facial 
recognition technology by South Wales Police in the United Kingdom.®” 


The Police used a facial recognition tool known as the AFR Locate, which extracts biometric data 
captured in a live feed from a camera, and compared the captured data to headshot photographs 
on a police watchlist. If a match is detected, the tool alerted a police officer who will then 
determine whether an intervention is needed. 


Edward Bridges, a civil liberties campaigner, was scanned by the facial recognition technology in 
Cardiff in December 2017 and again while attending a protest in March 2018. Although Mr Bridges 
was not included on a watchlist, he contended that given his proximity to the cameras, his image 
would have been recorded by the AFR Locate tool. 


Without making a factual finding on this issue, the Court acknowledged ‘scientific evidence that 
facial recognition software can be biased and create a greater risk of false identifications in the 
case of people from black, Asian and other minority ethnic (“BAME”) backgrounds, and also in the 
case of women’.°° 


The Court found that the use of facial recognition technology can breach human rights to privacy 
and equality or non-discrimination, and that the Police did not have lawful authority to use this 
tool. The Court identified two particular problems: 


The first is what was called the ‘who question’ at the hearing before us [who can be put on a watchlist 
for surveillance using the AFR Locate tool]. The second is the ‘where question’ [location of the 
deployment]. In relation to both of those questions too much discretion is left to individual police 
officers.°7 


The Court also found that the South Wales Police had failed to fulfil its positive duty to make 
enquiries regarding the potential discriminatory impact of the AFR Locate tool. Specifically, 

the Police had ‘never sought to satisfy themselves, either directly or by way of independent 
verification, that the software program in this case does not have an acceptable bias on grounds 
of race or sex.’ 8 


(b) Options for reform Some stakeholders agreed that facial recognition 
technology poses risks to human rights but queried 
A significant number of stakeholders backed a whether a limited moratorium would have the 
limited or partial moratorium on the use of facial desired effect given the technology is already in 
recognition in high-risk decision making. Many use.©? Others opposed even a limited moratorium, 
urged that regulation focus on the risk of mass on the basis that this could cause uncertainty 
surveillance.°° and impede the adoption of facial recognition 


technology, thereby disadvantaging Australian 


Some urged stronger regulation, including a ae : 
E E 6 6 citizens and industry.°4 


‘prohibition on unlawful and non-consensual 
harvesting of biometric data’. Others focused 
on broadening the technologies to which a 
moratorium would apply.652 
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The OAIC, which is the principal Australian 
Government authority on privacy, highlighted the 
existing privacy law safeguards that apply to the 
use of facial recognition, and welcomed further 
consultation on reform in this area.°° 


(c) International developments 


There is a growing movement for strict regulation 
on the use of facial recognition technology. For 
example, several state and municipal governments 
in the US have passed or are considering laws 
banning or restricting the use of facial recognition, 
especially in high-risk scenarios.°° Similarly, in 2020 
the German Government halted trials of the use of 
facial recognition by its police forces.%” 


In France, the relevant government regulator 
determined that schools were not permitted, under 
the GDPR, to install facial recognition on school 
premises in the manner proposed, and it also 
required the French Government to revise an app, 
‘Alicem’, which uses identity verification for access 
to 500 public services.°° 


However, in early 2020, the European Commission 
abandoned an earlier proposal for a moratorium 
on facial recognition technology. In its final White 
Paper on Artificial Intelligence, the European 
Commission stated that the use of Al for remote 
biometric identification purposes should be limited, 
and consideration given to the provisions of the 
GDPR and the European Union (EU) Charter of 
Fundamental Rights.°° 


In the EU, the revised Schengen Information System 
can use information from facial recognition and 
other biometric identification to help facilitate 

the return of migrants. In addition, the EU's 
iBorderCtrl project has piloted a prototype of the 
‘Intelligent Border Control’ system which uses a 
video ‘lie detection’ technology that has not shown 
high accuracy.®' The use of biometric data and 
certain technology more broadly in migration 
control has raised concerns about the impact on 
fundamental rights such as right to seek asylum.° 


Beyond government policy making, some 
technology companies have themselves taken 
important action in this area. In early 2020, 
Microsoft announced it would end all minority 
investments in companies that sell facial 


recognition technology given the sensitive nature 
of this technology. IBM also announced it would 
no longer invest in or supply facial recognition 
technology or analysis software. 6 Amazon made 
a similar announcement shortly after, introducing 
a one-year moratorium on police use of its facial 
recognition software, Rekognition.°© 


(d) Conclusion: a need for reform 


It is generally difficult and undesirable to regulate 
specific technologies. However, there are 
exceptions to this general principle.°° 


Governments tend to regulate high-risk activities 
and technologies more closely. This helps explain 
the comparatively strict laws that govern fields such 
as gene technology, aviation, healthcare and the 
energy industry. In these areas, regulation often 
applies both to the technology itself and how it is 
used. From a human rights perspective, the need 
for more prescriptive regulation will be greater 
where the use of a specific technology carries 
greater risks of harm to humans. 


It is critical, therefore, to assess the likely human 
rights impact of any particular use of facial 
recognition and other biometric technology. This 
requires answering three critical questions: 


e What specific type of biometric technology 
is to be used? For example, one-to-many 
facial recognition presents a greater risk to 
individuals’ human rights than one-to-one 
systems. 


In what context will the technology be used? 
In other words, what decisions will this 
technology be used to make? For instance, 
the use of one-to-many facial recognition 
technology in the context of policing or 

law enforcement presents a greater risk 

to human rights than the use of this same 
technology in a payment system at a café. 


What, if any, legal and other protections are 
already in place to address risks of harm? 


Existing laws—especially privacy and anti- 
discrimination legislation—offer some general 
protections in this area. However, for the following 
three reasons, the Commission has concluded that 
those existing protections are inadequate. 
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First, the Commission is concerned about the high 
rate of error, especially in the use of one-to-many 
facial recognition technology. The fact that these 
errors disproportionately affect people by reference 
to characteristics like their skin colour, gender and 
disability suggests that great caution should be 
exercised before this technology is used to make 
decisions that affect people’s legal and similarly 
significant rights. 


Secondly, in Australia and in many comparable 
jurisdictions, one-to-many facial recognition has 
been trialled in a range of high-stakes government 
and other decision making, including in policing, 
education and service delivery. Those trials 
generally have been in live scenarios, where any 
error resulting in a human rights infringement 
cannot be easily remedied, if it can be remedied at 
all. 


To date, existing legislation has not proven to 

be an effective brake on inappropriate use of 
facial and other biometric technology. Without 
effective regulation in this area, it seems likely that 
community trust in the underlying technology will 
deteriorate. One consequence could be distrust in 
both beneficial and harmful uses. 


Thirdly, the growth in facial recognition and 

other biometric technology, when coupled with 
other phenomena such as the growth in closed- 
circuit television (CCTV) cameras in public places, 
is contributing to an increased risk of mass 
surveillance. With limited legal protection against 
this cumulative impact, there is a real risk that 
Australians will cede their privacy incrementally, in 
ways that cannot be undone. 


For these reasons, targeted legislation is needed 
to prevent and address harm associated with 

the use of facial recognition and other biometric 
technology. As has been suggested by a number 
of leading experts, such legislation should prohibit 
certain uses of this technology, if human rights 
standards cannot be met.%” 


In 2021, the Council of Europe's Directorate 
General of Human Rights and Rule of Law issued its 
Guidelines on Facial Recognition. These guidelines 
identify a number of uses of facial recognition 

and related technologies, which merit strict legal 
limitation. It concluded: 
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The use of facial recognition for the sole purpose 
of determining a person's skin colour, religious 
or other beliefs, sex, racial or ethnic origin, age, 
health condition or social condition should 

be prohibited unless appropriate safeguards 

are provided for by law to avoid any risk of 
discrimination. 


Similarly, affect recognition can also be carried out 
with facial recognition technologies to arguably 
detect personality traits, inner feelings, mental 
health or workers’ engagement from face images. 
Linking recognition of affect, for instance, to hiring 
of staff, access to insurance, education may pose 
risks of great concern, both at the individual and 
societal levels and should be prohibited.°°° 


The Commission endorses this approach, which 
focuses especially on risks associated with 
particular types or uses of facial recognition. 

In Australia, the federal, state and territory 
governments are all engaged in activities that 

can involve high-risk uses of facial recognition 

and biometric technology, such as policing and 

law enforcement. In addition, all Australian 
governments share regulatory responsibility for the 
use of such technology by the private sector. 


Hence, all Australian governments should work 
cooperatively to introduce legislation that regulates 
the use of facial recognition and other biometric 
technologies. 


Until appropriate legislation is in effect, the 
Commission recommends the introduction of a 
limited or partial moratorium on the use of facial 
recognition and other biometric technology in Al- 
informed decision making (that is, decision making 
that has a legal or similarly significant effect). 
Particular attention should be given to high-risk 
contexts, such as the use of facial recognition 

in policing, in schools and in other areas where 
human rights breaches are more likely to occur. 


The development of legislation will involve a high 
degree of complexity, and so it should involve 
consultation with the states and territories, affected 
groups, industry and experts, as has been proposed 
for a similar US reform process.°”° 


9.5 Privacy reform 


RECOMMENDATION 21: The Australian 
Government should introduce a statutory 


cause of action for serious invasion of 
privacy. 


As biometric surveillance profoundly engages the 
right to privacy, there is also a need to consider 
reform to Australian privacy law itself. 


Australian privacy law is directed primarily towards 
information privacy. In this sense, current law 
offers limited protection to the right to privacy 
more broadly, and the Commission recommends 


Personal Data 


Car Registration 


[ Other H | 


the introduction of a statutory cause of action for 
serious invasion of privacy. 


(a) Options for reform 


Australian law prohibits the misuse of ‘personal 
information’ about an identified individual, 
including sensitive information (such as a 

person's health information, their racial or ethnic 
orientation, sexual orientation or criminal record) 
and credit information.®”’ The Australian Privacy 
Principles (APPs) in the Privacy Act 1988 (Cth) 
(Privacy Act), guide Australian Government agencies 
and some private sector organisations (APP 
entities) in how they collect, store, manage and use 
personal information, including in the context of 
data and data analytics. Principle 10, for example, 
requires APP entities to ‘ensure that the personal 
information that the entity collects is accurate, up- 
to-date and complete’.°” 
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The Australian Privacy Principles permit personal 
information that has been de-identified or 
anonymised to be processed for the primary 
purpose for which it was collected, based on the 
consent of the individuals concerned. However, 
technological advances are challenging this model 
of information privacy protection. For example, Al 
increasingly offers the capability of disaggregating 
a dataset made up of a conglomeration of de- 
identified data to reveal the personal information of 
specific, identifiable people. 


Data-processing technology is developing quickly 
with ever-increasing new uses of personal 
information being developed, many of which could 
not have been envisaged, let alone specifically 
consented to, at the point of collection. Certain 
potentially revealing information, including 
metadata, has been held by Australian courts not to 
fall within the parameters of the Privacy Act.§”3 


Three Australian law reform bodies—the Australian 
Law Reform Commission (ALRC) and its counterpart 
bodies in Victoria and New South Wales—have now 
recommended enacting a statutory cause of action 
for serious invasion of privacy.°” 


In 2014, the ALRC examined how new and 

serious invasions of privacy have arisen in the 
digital era—without legal protection.® The ALRC 
recommended the creation of a statutory cause 

of action for serious invasion of privacy, especially 
given the increased ‘ease and frequency’ of 
invasions of personal privacy that may occur with 
new technologies.°”¢ It recommended such a law to 
apply in two contexts: on intrusions upon seclusion, 
such as the physical intrusion into a person's 
private space; and misuse of private information, 
‘such as by collecting or disclosing private 
information’ about an individual.°’ 


By extending the protection of Australian law 
beyond ‘information privacy’, such reform could 
address some, though not all, of the concerns 
about how personal information can be misused 
in the context of facial recognition and other forms 
of biometric surveillance, and Al-informed decision 
making generally. 


More recently, the ACCC supported implementation 
of the ALRC’s recommendation in order to ‘increase 
the accountability of businesses for their data 


practices and give consumers greater control over 
their personal information’.°”® The ACCC noted that 
individual consumers are not able to bring direct 
actions for breaches of their privacy under the 
Privacy Act, or for serious invasions of privacy that 
result in financial or emotional harm.°” 


In response to the ACCC report, the Government 
announced a review of the Privacy Act.68° The 
Government stated that the review will identify any 
areas where consumer privacy protection can be 
improved and consider how to ensure Australia’s 
privacy regime operates effectively, and allow for 
innovation and growth of the digital economy.™' 
Due for completion in 2021, the review will consider 
the impact of new technologies, including Al, on 
whether Australian privacy law continues to be fit 
for purpose. 


Many stakeholders supported reform of privacy 
law, and in particular the creation of a statutory 
cause of action for serious invasion of privacy.°® 
Some stakeholders referred specifically to the 
desirability of implementing the recommendations 
of the ALRC and ACCC.%? 


Support for the proposal emerged from all sectors, 
including state and federal privacy regulators, 
privacy and legal experts, and technology 
companies. Stakeholders argued this would 
improve accountability, providing access to a 
remedy for some of the worst forms of privacy 
infringement, and increase the accountability of 
private organisations.° 


There was also a pragmatic acknowledgement that 
legislative reform would offer greater certainty in 
this area, rather than leaving change in this area to 
occur incrementally via the common law. 


Other stakeholders opposed this reform, or urged a 
more cautious approach. They referred to: 


e the increased risk of litigation 


* a potential chilling of innovation, with 
consequent social and economic impact on 
both the public and private sectors 


e the risk that a stronger privacy law could 
threaten the delivery of healthcare,®°> and 
could undermine press freedom.°°° 


(b) Conclusion personal information, to include interference with 
bodily and territorial privacy. This would more 


Several government-initiated inquiries have appropriately implement Australia’s obligations 
proposed the enactment of a statutory cause under Article 17 of the International Covenant 

of action for serious invasion of privacy. The on Civil and Political Rights, which requires states 
Commission agrees. to protect, in law, against the interference or 


attack on the ‘arbitrary or unlawful interference’ 
with an individual's ‘privacy, family, home or 
correspondence’.°°’ 


This reform would help combat the misuse and 
overuse of biometric technologies, and in fulfilling 
Australia’s human rights obligations regarding Al- 


informed decision making, given that Al frequently The Commission agrees with the OAIC that the 
involves the extensive use of personal information. statutory cause of action should be comprehensive 
and non-restrictive, and cover all intentional, 
reckless and negligent acts of privacy invasion by 
public and private entities. 


In practical terms, this reform would present a 
barrier to intrusive, wide-scale surveillance. It would 
extend privacy protection in Australian law beyond 
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10. Supporting effective regulation 


10.1 Summary 


This Part recommends the creation of an Al Safety Commissioner to support regulators, 
policy makers, government and business develop and apply policy, law and other 
standards in this area. The Commission considers that an Al Safety Commissioner could 
address three major needs. 


First, government agencies and the private sector are often unclear on how to develop 
and use Al lawfully, ethically and in conformity with human rights. An Al Safety 
Commissioner could provide expert guidance on how to comply with laws and ethical 
standards that apply to the development and use of Al. 


Secondly, regulators face the challenge of fulfilling their functions even as the bodies 
they regulate make important changes to how they operate. An Al Safety Commissioner 
could play a key role in building the capacity of existing regulators and, through them, of 
the broader ‘regulatory ecosystem’ to adapt and respond to the rise of Al. 


Thirdly, legislators and policy makers are under unprecedented pressure to ensure 
Australia has the right law and policy settings to address risks and take opportunities 
connected to the rise of Al. An Al Safety Commissioner could monitor trends in the use of 
Al here and overseas. This would help it to be a source of robust, independent expertise. 


As an independent statutory office that champions the public interest, including human 
rights, an Al Safety Commissioner could help build public trust in the safe use of 
Al—a goal the Australian Government is rightly seeking to advance.°°° 
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10.2 An independent Al Safety 
Commissioner 


The unprecedented rise in Al presents a once- 
in-a-generation challenge to develop and apply 
regulation that supports positive innovation, while 
addressing risks of harm. 


The Australian Government should establish an Al 
Safety Commissioner to support regulators, policy 
makers, government and business apply laws and 
other standards in respect of Al-informed decision 
making. 


(a) Value of an Al Safety Commissioner 


Stakeholders strongly supported the creation 
of an independent statutory authority to help 
government and the private sector manage 

the rapid growth in Al, especially in decision 
making.°°° Some saw the creation of an Al Safety 
Commissioner as a way to improve government 
cooperation and coordination in developing 
policy and regulation on the use of AI.® It was 
also suggested that this body could be a source 
of much-needed technical expertise to support 
existing regulators and others.°° 


Debate regarding the general idea of creating 
an Al Safety Commissioner centred on a key 
question: should this new body focus exclusively 
on addressing human rights and related risks, 
or should its remit be broader to include the 
promotion of innovation? 


A diverse group of stakeholders urged that the 
Al Safety Commissioner focus on protecting 
human rights,°? with several emphasising 

that strong human rights protections are the 
firmest foundation for public trust in the use 
of new technologies such as Al.°** Stakeholders 
also recognised the need for a broad mandate 
that includes the full spectrum of civil, political, 
economic, social and cultural rights—not solely 
the right to privacy.®° 


In supporting a focus on human rights, 
stakeholders observed the unequal impact of new 
technologies on some population groups.®® This 
was recognised also by some regulators such as the 
Office of the eSafety Commissioner.” Therefore, 
any new body should focus on protecting the rights 
of groups who can be at particular risk from the use 
of Al, such as people with disability.°* 


Some stakeholders argued that the Al Safety 
Commissioner should not be limited to 
promoting human rights—that is, it should also 
promote innovation.®? Others were concerned 
that there was an inherent conflict or tension 
between protecting human rights and promoting 
innovation.” 


There was also an intermediate position: the goal of 
innovation differs from the promotion of safety and 
human rights, but it is not necessarily irreconcilable 
with those other goals. In particular, a number 

of government and private sector organisations 
saw nothing unusual in a single government body 
promoting both innovation and human rights.” 
Some stakeholders warned that any action in this 
area should not stifle innovation or weaken the 
digital economy,” and highlighted the difficulty of 
regulating appropriately in this area.’ 


(b) Conclusion 


An Al Safety Commissioner could support 
regulators, policy makers, government and 
business apply laws and other standards in respect 
of Al-informed decision making. 


The use of Al, especially in momentous decision 
making, brings real human rights risks. For 

this reason, the primary focus of the Al Safety 
Commissioner should be to promote and protect 
human rights, with a special focus on groups at 
greatest risk of harm. 


The Commission acknowledges the legitimacy, 
indeed the importance, of promoting technological 
innovation and the growth of Australia’s digital 
economy.” To this end, responsible innovation 
should be central to the Digital Australia Strategy 
(see Chapter 3). 
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Moreover, ice arate be a T 10.3 Functio ns, powers 
commercial interest in promoting innovation . 

involving Al. By contrast, far fewer bodies are and ope rati on of Al Safety 
focused on promoting human rights, and safety Commissioner 


more broadly, in the use of Al especially in decision 
making. The Al Safety Commissioner should not 
play a direct role beyond those areas of focus—by, 
for example, promoting the economic benefits of RECOMMENDATION 22: The Australian 
innovation. Government should establish an Al 

Safety Commissioner as an independent 
statutory office, focused on promoting 
safety and protecting human rights in the 
development and use of Al in Australia. 
The Al Safety Commissioner should: 


In any event, a regulatory framework that 
effectively upholds human rights is more likely to 
foster, rather than stifle, responsible innovation. In 
particular, working to ensure that risks associated 
with Al are effectively understood and addressed 


would help create a stable regulatory environment, (a) work with regulators to build 
which can help to drive investment. their technical capacity regarding 

o OE SENEST the development and use of Al in 
Similarly, by strengthening institutions areas for which those regulators have 


responsible for protecting the public, the Al responsibility 
Safety Commissioner could contribute to a firm 
foundation of public trust in how Al is developed 
and used in Australia. This is consistent with 
growing concern in Australia and internationally 
that regulation needs to be more effective in 
protecting human rights in Al-informed decision (c) provide independent expertise 
making.” relating to Al and human rights for 
Australian policy makers 


(b) monitor and investigate 
developments and trends in the use 
of Al, especially in areas of particular 
human rights risk 


(d) issue guidance to government and 
the private sector on how to comply 
with laws and ethical requirements in 
the use of Al. 
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RECOMMENDATION 23: TheAl Safety 
Commissioner (see Recommendation 22) 
should: 


(a) be independent from government 
in its structure, operations and 
legislative mandate, but may be 
incorporated into an existing body or 
be formed as a new, separate body 


(b) be adequately resourced, wholly 


or primarily by the Australian 
Government 


(c) be required to have regard to the 
impact of the development and use 
of Al on vulnerable and marginalised 
people in Australia 


(d) draw on diverse expertise and 
perspectives including by convening 
an Al advisory council. 


The remainder of this Part considers the optimal 
functions, powers and operation of an Al Safety 
Commissioner. 


(a) Building the capacity of existing 
regulators 


We have always made predictions and decisions. 
Increasingly, it is Al that is driving those predictions 
and decisions. 


This presents a challenge for regulators: as Al 
enables new ways to analyse data, to make 
predictions and ultimately to form decisions, 
regulators must continue to fulfil their functions 
even while the bodies they regulate are changing 
how they act. 


The Commission considers that the primary role 
of the Al Safety Commissioner should be to assist 
regulators manage a major transition towards 
the use of Al in the public and private sectors. It 
would build their capacity to fulfil their regulatory 
functions amid the rise of Al-informed decision 
making. 


(i) The challenge of Al for regulators 


The Al Safety Commissioner should work with 
regulators to provide direct training, focused on 
how Al is used in the areas of activity that each 
individual regulator has responsibility for, as well 
as building an understanding of the typical ‘lifecycle 
of Al’. This would help regulators assess where 
problems are most likely to arise. 


Take, as an example, how financial service 
providers, like banks, make lending decisions. This 
is not a new activity: banks have always assessed 
the creditworthiness of people who apply for 
loans, and made decisions accordingly. The critical 
difference is the method by which banks make 
those decisions is changing—with that assessment 
increasingly made using Al. 


The challenge for regulators is to apply the relevant 
rules and laws in this changing environment. 
Regulators generally recognised the need for 
increased capacity building, coordination and 
cooperation to respond to the rise of Al in their 
areas of responsibility.”°° For example, the OAIC 
stated: 


[Rlegulators require technical expertise to enforce 
regulatory schemes as the use of technologies 
such as Al becomes increasingly prevalent in the 
public and private sectors. Technical capacity- 
building within regulators would further support 
regulatory efforts. Also, a body could act as a 
central source of technical expertise and capability 
to assist agencies and regulators in addressing 
these challenges. We consider that the UK Centre 
for Data Ethics and Innovation (CDEI), referred 

to in the discussion paper, is a model that has 
merit.7°” 


It was suggested that an Al Safety Commissioner 
could work with individual regulators to understand 
how Al is used in their respective settings.” Some 
of these settings are highly specialised and, in some 
areas, error can result in significant human rights 
violation. The use of Al to make decisions in the 
criminal justice and law enforcement area is an 
important case in point.” 
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(ii) No need for a dedicated regulator of Al 


The Al Safety Commissioner should not have a 
broader regulatory role, beyond supporting existing 
regulators. Al is a cluster of technologies that can 
be used in widely divergent ways in almost limitless 
contexts. Some uses of Al engage human rights, 
while others do not. A focus on regulating Al as 
technologies would not be as effective as regulating 
uses of Al in particular contexts. Regulators will be 
better placed to respond to the seismic changes 
prompted by the rise of Al if they are supported by 
the Al Safety Commissioner recommended in this 
Part. 


Multiple stakeholders recognised the burden 

on existing regulators in protecting people from 
risks of harm associated with Al. To this end, they 
welcomed the idea of a new body that would assist 
regulators in bearing that burden.7!° 


Some stakeholders cautioned against an Al 

Safety Commissioner moving beyond the role of 
supporting regulators, to become a regulator itself, 
and there was general agreement that neither the 
Al Safety Commissioner nor any other body should 
be the single regulator for all uses of Al.” 


For example, the Digital Industry Group, which 
represents technology companies including Google 
and Facebook, expressed caution about the risk 

of creating a new Al regulator, arguing that most 
problems that occurred with Al were the contextual 
application of Al in a particular sector.’" 


(b) Monitoring, investigation and policy 
expertise 


The Al Safety Commissioner should be a source 
of trusted expertise on Al policy for government. 
The need is clear: many of the issues in this area 
are technical and complex and require cross- 
disciplinary expertise. 


The Al Safety Commissioner could provide expert 
advice on issues such as: 


e Australian law and policy reform, including 
the application of human rights law 


e voluntary standards and ethical frameworks 


e the direction of advances in relevant areas 
of scientific discovery. 
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In doing so, the Al Safety Commissioner could 
draw on material from Australia and elsewhere— 
including from regulators, government, law, 
industry, academia and civil society. 


This would respond to a need recognised 
internationally and in Australia. Anumber of major 
international bodies—such as the OECD, the World 
Economic Forum, and the Council of Europe—have 
identified a need for governments to be better 
informed about the implications of Al, including its 
risks and opportunities.’"? The Australian Council of 
Learned Academies has specifically pointed to the 
need for Australian policy makers to have access to 
frank, expert advice on precisely how the rise of Al 
and related technological developments are causing 
change that the Government needs to be aware of 
and respond to.’"4 


Stakeholders in Australia also emphasised the need 
for an expert body to advise our Government on 
Al, with a view to developing good law and policy 

in this area, and specifically to promote human 
rights in ways that do not stifle positive, responsible 
innovation.’'> It was suggested that the proposed 
Al Safety Commissioner would be well placed to 
monitor national and international developments 
to help promote best practice.’'° In this way, the 
body could contribute to long-term policy on Alin 
Australia.” 


(i) Expertise on developments and trends in Al 


To provide the necessary guidance and advice, the 
Al Safety Commissioner would need to monitor 
and report on how Al is being developed and used 
in decision making in Australia and internationally, 
especially in areas where human rights are likely to 
be especially at risk. 


The monitoring and reporting function is important 
due to the rapid development and growing use of 
Al. Given that some forms of Al tend to be opaque, 
as there is a lack of transparency in how the 
algorithm makes decisions, there is a heightened 
need for specific expertise in monitoring Al 
developments, especially in relation to the 
operation of Al-informed decision-making systems, 
which can be difficult to assess.7'8 


The need for good coordination within government 
was a recurring theme in the Commission's 
consultation process.’'? The Al Safety Commissioner 
could play a role in fostering collaboration with 
other Australian Government bodies that have 
relevant expertise, such as the ACCC, the Office 

of the eSafety Commissioner and the OAIC. 7% It 
could also bring to bear input from experts outside 
of government, and from groups that are not 
frequently consulted on policy issues relating to Al. 


Some stakeholders pointed to a need for broader 
capacity building on Al across government. KPMG 
Australia, for example, suggested that the Al Safety 
Commissioner could play a role in developing 

an Al curriculum for government officials, which 
would help them understand the opportunities and 
risks.” 


More broadly, the Al Safety Commissioner could 
draw on the expertise of international counterparts, 
including intergovernmental organisations such as 
the OECD and organisations such as the UK Centre 
for Data Ethics and Innovation.’ 


(ii) Investigative powers 


One specific concern raised by some stakeholders 
related to the lack of transparency in how Al and 
algorithms are used by government and others.’ 
This problem could be addressed by giving the 

Al Safety Commissioner a function to monitor 
developments in AI.” Monitoring the use of Al 
could range from assessing individual areas of 
human rights concern on a case-by-case basis, to 
systemic monitoring of Al trends. 
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The Al Safety Commissioner could obtain much of 
the information it needs through publicly available 
sources or with the cooperation of industry. 
However, much of the research and development 
in this area occurs in commercial settings where 
strict limits are set on how information may be 
disclosed.” The Al Safety Commissioner should 
remain abreast of change in this area, especially as 
it relates to human rights. 


Some urged that the Al Safety Commissioner be 
given investigative powers to inquire into practices 
of particular concern.’ The NSW Bar Association 
advocated going further, suggesting that if a 
statutory office is established, it should possibly 
have enforcement powers.’’ Similarly, the NSW 
Council of Civil Liberties suggested providing 
additional powers to investigate and detect bias in 
algorithms through system audits, accompanied by 
legal enforcement and authority to impose a ban or 
a fine.’ 


The Commission recommends that the Al Safety 
Commissioner be given powers to investigate or 
audit the development and use of Al (including 
algorithms) in some circumstances to identify 

and mitigate human rights impacts. This is a 
power available to expert government bodies 

in other jurisdictions, such as the Information 
Commissioner Office in the UK, and it has been 
called for by civil society groups and experts in the 
Toronto Declaration.” While it is anticipated that 
investigative powers would need to be exercised 
only sparingly, they would be an important tool 
available to provide a clear picture of the direction 
of Al in this country. 


Others opposed the Al Safety Commissioner 
focusing on the detail of how algorithms and Al 
work, given the risk that such monitoring could 
lead to the publication of sensitive commercial 
information. Being required to disclose such 
information could deter companies expanding or 
initiating investment in Australian Al which could, in 
turn, affect the quality and diversity of Al products 
available in Australia.” 


(iii) An Al advisory council 


One way of ensuring the Al Safety Commissioner 
is well informed about developments in Al would 
be for it to convene an advisory council on Al. Such 


an Al advisory council could have similar functions 
to those of the proposed National Data Advisory 
Council in relation to sharing and use of public 
sector data, and assist the Al Safety Commissioner 
in performing its functions.’?" 


The advisory council could include senior 
representatives from bodies such as the Australian 
Competition and Consumer Commission, the Office 
of the Australian Information Commissioner, the 
Office of the eSafety Commissioner, the National 
Data Commissioner, the Australian Human Rights 
Commission, the Digital Transformation Agency, 
the Department of Industry, Science, Energy and 
Resources, Australia’s Chief Scientist, CSIRO, and 
the Australian Statistician. 


The composition of the advisory council would 
mean that high-level external expertise from 
sectors relevant to Al and human rights could be 
accessed in areas relating to building capacity 

of existing regulators, monitoring the use of Al 

and providing policy advice. Formal access to this 
expertise would enable the Al Safety Commissioner 
to play a practical role in helping regulators and 
other key government agencies coordinate their 
activities. 


(c) Guidance on responsible use of Al 


Where government and the private sector are 
considering the use of Al in decision making, 
they would also benefit from technical guidance 
on complying with existing laws and ethical 
frameworks, especially as they relate to human 
rights.7*2 The Al Safety Commissioner, with cross- 
disciplinary expertise, would be well placed to 
develop such materials. 


(i) Aneed for guidance 


Guidance could provide practical tools on a range 
of areas including: 


e the application of existing consumer 
protection, privacy and human rights laws 
to Al-informed decision making 


e the use of voluntary standards and ethical 
frameworks 


e the use of human rights impact assessment 
and design tools in this area. 


For example, government agencies would benefit 
from guidance on how to fulfil their obligations to 
provide reasons for administrative decisions, when 
those decisions are made using Al. In particular, 
such guidance could deal with how a technical 
explanation could be provided.” 


Guidance could also include practical tools that 
support a co-regulatory approach. This could 

assist public and private sector bodies comply with 
human rights standards for Al, and conduct human 
rights impact assessments.” 


The Commission's 2020 technical paper on 
algorithmic bias is an example of the sort of 
guidance that might be of assistance in this regard. 
That paper, Using artificial intelligence to make 
decisions: Addressing the problem of algorithmic bias, 
was the product of a collaboration with Gradient 
Institute, Consumer Policy Research Centre, CHOICE 
and CSIRO’s Data61.735 


Stakeholders from the public and private sectors 
identified a need for expert advice on the use of Al 
in ways that uphold human rights.” Stakeholders 
also recognised the need for guidance to promote 
a clearer understanding of how existing laws and 
policies apply to the development and use of Al.’%” 


Some stakeholders identified a need for practical 
guidance in particular contexts where Al is being 
used, including in business and health.” For 
example, KPMG Australia noted that business 
would benefit from practical guidance on applying 
human rights principles in relation to Al-informed 
decision making.” The SAS Institute Australia 
identified a desire from the technology community 
for ‘clear and simple guidance and a meaningful 
“translation” of how to consider legal requirements 
and rules’.’4° 
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(ii) Specific recommendations where guidance 
is needed 


In Part B, the Commission recommends a 
number of important activities that the Al Safety 
Commissioner would be suitable to perform— 
especially in issuing guidance. These are: 


conducting an audit of all current or 
proposed use of Al-informed decision 
making by or on behalf of Government 
agencies”! 


developing guidance for government 
and non-government bodies on how to 
generate reasons, including a technical 
explanation, for Al-informed decisions’ 


issuing guidance for the private sector on 
how to undertake human rights impact 
assessments”? 


convening a multi-disciplinary taskforce on 
Al-informed decision making to consider 
how to embed human rights throughout 
the Al life cycle” 


developing a tool to assist private sector 
bodies undertake human rights impact 
assessments in developing Al-informed 
decision-making systems”° 


reviewing Australian law, policy and 
guidance relevant to government 
procurement of Al-informed decision- 
making tools’ 


issuing guidance to the private sector on 
good practice regarding human review, 
oversight and monitoring of Al-informed 
decision-making systems.””” 


In time, there could also be a role for the Al Safety 
Commissioner in broader public education and 
awareness raising about Al. 


(d) Resourcing, structure and expertise 


The Commission recommends that the Al Safety 
Commissioner be an independent appointment, 
with core funding from the Australian Government 
to help ensure independence and secure the 
public's trust. There could be opportunities 

for other income sources from industry or the 
community, with appropriate protections to prevent 
conflicts of interest or inappropriate commercial 
pressures. 


The Al Safety Commissioner should have access 
to expertise across a broad range of professional 
disciplines, including engineering, science, 
technology; law and business; human rights and 
ethics; and social science. This range of expertise 
is necessary due to the complexity of Al-informed 
decision-making systems, and the need for a 
sophisticated understanding of the operation of 
such systems. 


While it is important that the Al Safety 
Commissioner be an independent statutory 
authority, the Commission remains agnostic on 
whether it would be better to incorporate it into an 
existing body or to create a separate entity. 


There are successful examples of each of these 
models. For example, the Office of the eSafety 
Commissioner was established as a standalone 
body under the Enhancing Online Safety Act 2015 
(Cth).”“8 The eSafety Commissioner is supported 
by the ACMA, with all staff except for the 
Commissioner employed by ACMA.74? 


Another model, where an independent 
commissioner is integrated entirely within an 
existing statutory body is the National Children’s 
Commissioner in the Australian Human 

Rights Commission.” The National Children’s 
Commissioner performs its functions on behalf of 
the Commission and has a responsibility to work 
collegially with other Commissioners. Benefits of 
this approach include being able to have formal 
access to expertise within the Commission on a 
range of human rights matters that are of relevance 
to children.’*! 


A decision about where the Al Safety Commissioner 
is located should be informed by a transparent 
consultation process facilitated by the Attorney- 
General’s Department. Consultation with industry, 
regulators and other government agencies would 
help ensure this decision minimises any duplication 
of roles and activities between the Al Safety 
Commissioner and other bodies. It would also help 
build acceptance and understanding of the role of 
the new Al Safety Commissioner in industry and the 
community. 


Stakeholders noted the need for adequate 
resourcing of an Al Safety Commissioner.’* 
Responsible Technology Australia suggested 
an industry levy could be one way of securing 
appropriate independence.’*? 


Stakeholders identified a need for specialised 
technical expertise on Al, and in-depth sector- 
specific knowledge across different settings where 
Al was deployed.’** The Australian Academy of 
Sciences, for example, considered that both cross 
disciplinary and interdisciplinary expertise are 
needed to deal with the complex problems raised 
by AI.755 


Zi 


A strong theme throughout the consultation 

was that establishing an Al Safety Commissioner 
presented an opportunity for civil society 

input on law and policy relating to Al. Some 
stakeholders emphasised the need for the Al Safety 
Commissioner to play a role in protecting the 
human rights of groups that experience structural 
and individual discrimination.’”°° 


Stakeholders generally agreed that the Al Safety 
Commissioner should be independent, with 
legislative backing.” However, there were differing 
views on how this should be achieved. Some 
stakeholders focused on avoiding duplication of 
roles and responsibilities with other regulators 
and urged that a new body focus on collaboration 
with existing regulators.”* Some suggested that 
the Al Safety Commissioner be incorporated into 
an existing body that focuses on human rights and 
consumer interests such as the Commission, OAIC, 
ACCC or ASIC.759 


The OAIC observed that an existing agency could 
be well placed to take the lead in implementing the 
national strategy (Recommendation 1). This could 
be done ‘in collaboration with a technical body 
such as Data61’, with a view to ‘driving a whole- 
of-Government, economy-wide approach to Al 
technology’.’©° 
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ll. The right to access technology 


11.1 Summary 


This Part of the Final Report considers how people with disability are experiencing the 
rise of new technologies. It recommends practical ways of promoting more equal access 
to the benefits of new technology for people with disability. 


All members of society use technology to participate in almost every aspect of individual 
and community life. This includes in areas that are fundamental to the enjoyment of our 
human rights, such as education, government services and employment. As a result, our 
human rights depend on being able to access and use technology. 


New technology can enable the participation of people with disability as never before— 
from the use of real-time live captioning to reliance on smart home assistants. But 
inaccessible technology brings a heightened risk of exclusion. 


Poor design, such as cashless payment systems that are inaccessible to people who 
are blind, can cause significant harm, reducing the capacity of people with disability 
to participate in activities that are central to the enjoyment of human rights, and their 
ability to live independently. 


Two types of ‘access’ are referred to throughout this part of the Final Report. The 
first involves being able to obtain goods, services and facilities that use a particular 
technology. The second type, known as ‘functional access’, involves being able to use 
goods, services and facilities that use a particular technology.’®' 


Reflecting the feedback from stakeholders, this Part of the Report focuses on 
technologies that are used primarily for communication. The Report uses the term 
‘Digital Communication Technology’ to refer to technologies associated with information 
and communication technology (ICT), connected devices and the Internet of Things (loT), 
virtual reality (VR) and augmented reality (AR). 


This first chapter of Part D focuses on the right to access technology, and especially 
Digital Communication Technology. The later chapters in this Part consider the functional 
accessibility of technology (Chapter 12), accessible broadcasting and audio-visual 
services (Chapter 13), the experience of people with disability in obtaining technology 
(Chapter 14), and design, education and capacity building in relation to technology 
(Chapter 15). 


1. Accessing technology is an enabling right for people with disability. 
2. Many people with disability encounter barriers in accessing Digital 
Communication Technologies. 


3. Law and policy reform is needed to improve functional access to Digital 
Communication Technologies. 

4. A‘human rights by design’ strategy can improve the functional accessibility 
of Digital Communication Technologies. 
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Addressing disability 
discrimination 


New Disability 
Discrimination Act 
Standard 


Industry action 


Concessional 
NBN 
broadband 
for people 
with disability 


Inquiry into 
industry 
compliance 
with 
accessibility 
standards 


Design, education & 
capacity building 


New voluntary 
standards 


Human rights by design 
in curricula 


Accessible technology in 
new National Disability 
Strategy 


Government 
action 


News, information 


& entertainment 


Audio 
vals description & 
procurement captioning 
rules promote 
accessibility 

Australian 
Governments comply 
with accessibility 
standards: policies & Accessible 
targets emergency 
announcements 


Accessible tech 
under the NDIS 


11.2 The rig ht to access States Parties to the CRPD must take appropriate 
technol ogy measures to identify and eliminate barriers to 


people with disability accessing technology, 
including by implementing minimum accessibility 
standards, and promoting the design and 
development of accessible technology.” As 
discussed in Chapter 12, a number of Australian 
laws, policies and guidelines go some way to 
implementing this obligation. 


Australia is a party to the Convention on the Rights 
of Persons with Disabilities (CRPD). The CRPD aims 
to ‘promote, protect and ensure the full and equal 
enjoyment of all human rights and fundamental 
freedoms by all persons with disabilities’.” Article 
9 focuses on accessibility and sets out the right 

of people with disability to access ICT and other The following CRPD articles are particularly relevant 
technology open or provided to the public.” This to the right of people with disability to access 

right is referred to in this Part as the ‘right to access technology. 

technology’.” 
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Box 9: Key provisions of CRPD:’ 


Convention principles Article 3 


a) Inherent dignity, individual autonomy, freedom of choice and 
independence; 

b) Non-discrimination; 

c) Full and effective participation and inclusion in society; 

d) Respect for difference and acceptance; 

e) Equality of opportunity; 

f) Accessibility; 

g) Equality between men and women; 

h) Respect for evolving capacities for children and their identities. 


Accessibility Article 9 

People with disability have the right to access all aspects of 
society on an equal basis with others including the physical 
environment, transportation, information and 
communications, and other facilities and services provided 
to the public. 


Living independently and being included in the 
community Article 19 


People with disability have the right to live independently 
in the community 


Freedom of expression and opinion, and access 
to information Article 21 


People with disability have the right to express themselves, 
including the freedom to give and receive information and 
ideas through all forms of communication, including 
through accessible formats and technologies, sign 
languages, Braille, augmentative and alternative 
communication, mass media and all other accessible 
means of communication. 
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(a) The right to access technology as an 
enabling right 


Stakeholders consistently described access to 
technology as an enabling right.’”*” For example, 
the Australian Communications Consumer Action 
Network (ACCAN) stated: 


Access to technology can offer expanding 
opportunities to people with disability. With 
greater access to accessible technology comes 
greater inclusion within society and more equal 
enjoyment of human rights, including for instance 
more inclusive workplaces, better access to 
education (including lifelong learning), and greater 
participation in cultural life, in recreation activities, 
leisure and sport.’ 


As Article 9 of the CRPD reflects, access to 
technology can ‘enable persons with disabilities to 
live independently and participate fully in all aspects 
of life’. It enables the enjoyment of other CRPD 


` 


» 


rights and is a key underlying principle,”°° because 
it is ‘a vital precondition for the effective and equal 
enjoyment of civil, political, economic, social and 
cultural rights by persons with disabilities’.””° 


Accessible technology can be compared with other 
enabling rights, such as the right to education, as it 
helps build skills, capacity and confidence to help 
people achieve other rights.” 


(b) Inaccessible technology and impact on 
other human rights 


Digital Communication Technologies are pervasive 
and indispensable throughout our community, 
including in workplaces, for education, healthcare 
and as a means of communicating.’”* Stakeholders 
made clear that lack of access to these technologies 
can limit enjoyment of CRPD rights.” Some 
illustrations are set out in Table 1 below. 
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Table 1: CRPD-protected rights engaged by inaccessible technology 


The right to work (art 27) The right to work is compromised when the technologies required 
for work are inaccessible. This problem is exacerbated by the near 
ubiquity of ICT in the workplace. For example, more and more job 
advertisements appear primarily or solely online, which disadvantages 
those who cannot access such websites. Similarly, it was reported 
that ICT routinely used in employment settings—such as for content 
management and internal human resources and finance systems—is 
not accessible. This creates a barrier for people with disability working 
in those environments. 


The right to freedom of This right can be constrained by difficulties for people with disability 
expression and opinion (art 21) accessing ICT hardware, easy-to-read online information and websites. 


The right to education (art 24) This right is limited when accessible educational materials are not 
provided. Inaccessible technologies can have a significant effect on the 
education of children with disability. 


The right to privacy (art 22) Privacy can be breached when technology tools inappropriately collect 
personal information from people with disability. 


Political rights (art 29) This right can be compromised when accessible ICT hardware is not 
provided at voting stations and information portals. 


The right to highest attainable This right can be compromised when the cost of assistive technology is 
standard of health (art 25) prohibitive or a technology is not available for people who live in rural 
or remote areas. 


The rights to equality before These rights can all be negatively affected by the use of predictive data, 


the law (art 5), equal legal especially where it leads to algorithmic bias affecting groups such as 

capacity (art 12) and effective people with disability.’”* These rights may also be compromised when 

access to justice on an equal access to justice requires online communication which is inaccessible 

basis with others (art 13) to a person with disability. 

The right to cultural life (art This right can be limited where television programs, subscription 

30) services, films, online content and theatre are not captioned or audio 
described. 

Situations of risk and States must take all necessary measures to protect and safeguard 

humanitarian emergencies people with disability in situations of risk and emergencies. This brings 

(art 11) special obligations during the COVID-19 pandemic and in natural 


disasters, such as bushfires. 
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11.3 The COVID-19 pandemic 


The Commission's final stage of public consultation 
occurred in the first half of 2020. People with 
disability and their representatives reflected on 
their experience of the direct and indirect effects of 
the COVID-19 pandemic. 


A consistent message was that COVID-19 has 
highlighted and exacerbated existing inequalities, 
including in regard to inaccessible Digital 
Communication Technologies.” Prior to the 
pandemic, the National Association of Community 
Legal Centres observed: 


[E]xisting inequalities are often perpetuated 
through access to technology, and access to 
technology is increasingly central to engagement 
with essential services and opportunities.’”° 


A direct consequence of the pandemic has been 

to increase the reliance on digital technology for 
participation in all aspects of society, including 
education, employment and healthcare. This has 
heightened the importance of accessible technology 
for people with disability. 


Some specific concerns raised by people with 
disability and their representatives included the 
following.” 


(a) Internet access 


Restrictions associated with the COVID-19 
pandemic increased the need for reliable and 
affordable broadband for people with disability. 
Online became the sole or principal way of 
accessing carers, health supports, workplaces and 
education facilities during pandemic restrictions. 


Stakeholders also referred to the problems of 
isolation, especially for people with disability who 
received less in-person support as a result of 
government lockdowns or advice to reduce social 
interaction. Instead, many were forced to rely more 
on remote platforms to stay connected with people. 


Many stakeholders observed that the sudden 
transition to working, studying and connecting via 
digital platforms increased the urgency of problems 
they had been experiencing before COVID-19. The 
existence of an inaccessible digital platform was 
not a new problem but during COVID-19 often 

the digital platform became the only way that a 
service provider, health professional, workplace or 
education facility was communicating. 


Examples included: 


e barriers for people with disability in carrying 
out their employment duties when they 
were suddenly instructed to ‘work from 
home’, due to inaccessible software and 
digital platforms 


people who are engaging with the justice 
system being unable to access legal services 
or participate in a court matter via videolink 
which is inaccessible to a person with 
disability.’” 


(b) Emergency and public service 
announcements 


Federal, state and territory governments have 
communicated vital information related to the 
pandemic through public service announcements 
and special broadcasts. However, some of this 
information has been communicated in ways that 
are inaccessible for some people with disability. 


For instance, some early features of the COVIDSafe 
app were not functionally accessible for people who 
are blind or who have a vision impairment (though 
these features were later included via software 
updates). In addition, as discussed in greater detail 
in Chapter 13, some emergency broadcasts were 
not accessible for many people with disability. This 
included some televised community information 
about safe hand washing and mask wearing which 
did not include audio description, captions or 
Auslan. 
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(c) An opportunity for positive change 


Stakeholders also shared some positive stories 
emerging from the shift to digital platforms during 
the COVID-19 pandemic.’”? A representative 

from one disability group noted that some of 

their members had previously requested more 
flexibility in their working arrangements, which 
had been denied. However, with more people now 
working from home, many employers took this 
opportunity to reconsider these arrangements, 
providing greater flexibility for people with disability 
to structure their care supports, personal and 
professional lives.’°° 


One disability group representative noted: 


This pandemic has shown everyone that we need 
good policies in place before an event like this— 
we need to plan for crisis-driven scenarios. It’s 

not good enough to hope that people will do the 
right thing, we need enforceable law for accessible 
technology." 


Given the dramatic and rapid shift from in-person 
communication to digital platforms, accessible 
technology is more urgent than ever. This change 
has highlighted the existence of inaccessible goods, 
services and facilities, and reinforces the imperative 
to promote and protect the right to access 
technology. 


11.4 Importance of accessibility 


People with disability have diverse experience with 
Digital Communication Technologies. However, 

as these technologies are integrated into almost 
every conceivable area of life, the importance of 
accessibility grows ever greater. 


When accessible, these technologies—and the 
goods, services and facilities in which they are 
used—enable people with disability to enjoy greater 
independence and participation in all areas of 

life. When inaccessible, they present barriers to 

the most fundamental human rights, including to 
education, employment, health and more. 


The right to access technology, and 
especially Digital Communication 


Technology, is an enabling right that 
supports people with disability to realise 
a range of political, economic, social and 
cultural rights. 


The following four chapters explore some key 
implications that flow from this finding, and set out 
recommendations for reform. In particular, these 
recommendations would assist Australia to fulfil its 
obligation to implement CRPD rights.”8? 


le. Functional accessibility 


12.1 Summary 


This chapter considers the functional accessibility of goods, services and facilities that 
rely on Digital Communication Technology. 


Something is functionally accessible if it accommodates the needs associated with 
a person's disability. This form of technology tends to be designed for the entire 
community, not solely people with a particular disability. 


By contrast, ‘assistive technology’ is specifically designed to support people with a 
particular disability to perform a task. An example of assistive technology is a screen 
reader, which can assist a person who is blind, or who has a vision impairment, to read 
the content of a website. 


Where problems arise in goods, services and facilities that use Digital Communication 
Technologies, they tend to relate to the user interface being designed in a way that 
excludes people with one or more disabilities.” For example, a person who is blind 
cannot read information on a visual display like a typical computer screen. 


Different disabilities bring different accessibility requirements.’** Hence, functional 
accessibility is sometimes described as a match between the user's individual 
accessibility needs and the user interface of the product or service.” 


The recommendations in this chapter aim to improve functional accessibility in three 
ways: 


1. the creation of a new Disability Standard, focused on Digital Communication 
Technology, under the Disability Discrimination Act 1992 (Cth) (DDA) 


2. amending government procurement rules to require accessible goods, services 
and facilities 


3. improving private sector use of accessible Digital Communication Technology. 


l2.2 Functional accessibility in practice 


Functional accessibility needs to be incorporated in the design of both the hardware and 
software, and in any updates or new iterations, of goods, services and facilities that use 
Digital Communication Technologies. 


For example, a particular smartphone may be accessible, in the sense that it can be 
operated by people with disability. However, some applications that people buy to use 
on the smartphone may not be accessible. Industry stakeholders observed that the 
online stores where consumers obtain these applications play an important gatekeeping 
role: they can mandate accessibility, security and other features of applications that they 
make available. Ultimately, however, application developers are primarily responsible for 
making applications accessible.’°° 
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When an accessibility problem relates to a Digital Communication Technology that is used in many types of 
product and service, all of the resultant products and services can be rendered inaccessible.” An example 
of this phenomenon is where a device requires a passcode or PIN to be entered by using a touchscreen 
(also known as ‘PIN on Glass’ or POG). Unless specially modified, this technology is generally inaccessible for 
people who are blind or have a vision impairment, because it is impossible for them to know where to enter 
the passcode. 


People with disability and representative groups outlined many examples of goods, services and facilities 
that use Digital Communication Technology, and which are not functionally accessible.’® 


Table 2: Examples of functional accessibility problems 


Digital 
Communication 
Context Technology Limits on functional accessibility 
Domestic and Connected and Some digital home assistants may be more accessible for people who 
personal smart digital are blind or have a vision impairment, but those that require voice 
devices input are inaccessible for people with a voice communication disability. 
Whitegoods Products with touch screens are sometimes inaccessible for people 
and household who are blind or have a vision impairment. Connectedness through loT 


electrical devices may help address this problem—at least for people with disability who 
are able to connect online. 


Employment ICT Some content management systems, client referral systems, internal 
human resources and finance, and business technologies are 
inaccessible for people with various types of disability. 


News and Broadcasting, A high proportion of broadcasting and video content available to the 
entertainment video and online Australian public is inaccessible to people who are deaf or have a 
services content hearing impairment. 


There is currently no minimum requirement for audio description, for 
people who are blind or have a vision impairment, in respect of free- 
to-air television, on-demand and subscription broadcasting and online 
content. 


Emergency and public service announcements are inaccessible for 
people who are deaf or have a hearing impairment where automatic 
captioning is unavailable or inaccurate, and Auslan interpreters are not 
provided or visible on the screen. 


Online Web accessibility | Web accessibility has improved through wider use of accessibility 

information guidelines, but there remain many inaccessible websites, webpages 
and online services, especially for people with print disability. 
Inaccessible security tools such as CAPTCHA, which are incorporated 
into otherwise accessible webpages, can render the whole content 
inaccessible for the user. 
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The Commission heard many examples of 
digitisation making a service, which was previously 
accessible, inaccessible for people with disability.’°° 
These include: 


* Some machines used for credit card 
payment in shops require payment via a 
POG touch screen, and automated self- 
service and information kiosks can be 
inaccessible or more difficult to use for 


Interactive voice response software, 
which is often used on service telephone 
lines, is inaccessible for some people with 
intellectual disability who require human 
customer support.’ 

The primary means of accessing some 
essential government services, such as 
My Health Record, My Aged Care and 

the National Disability Insurance Scheme 


(NDIS), is now online. This can present 
barriers for people with limited or no 
Internet coverage or computer literacy.” 


people with a vision impairment.” 


Automatic airline check-in using passport 
facial recognition technology can present 
barriers for people with eye or facial 


differences.” 12.3 Anew Standard under the 
Smartphone transport applications, which Disabil ity Discrimination Act 
replace information in Braille or talking text 
at transport interchanges, are difficult for 
people who do not have access to mobile 
phone applications or are not confident 
digital users.” 


RECOMMENDATION 24: The Attorney- 
General should: 


(a) develop a Digital Communication 
Technology Standard under section 
31 of the Disability Discrimination Act 
1992 (Cth), and 


(b) consider other law and policy 


reform to implement the full range 
of accessibility obligations regarding 
Digital Communication Technologies 
under the Convention on the Rights of 
Persons with Disabilities. 


In doing so, the Attorney-General should 
consult widely, especially with people with 
disability and the technology sector. 


The Commission recommends the creation of a 
new Digital Communication Technology Standard 
under s 31 of the DDA. 


A broad range of stakeholders—including people 
with disability, community and advocacy groups, 
and academic experts—advocated strengthening 
disability discrimination law in respect of Digital 
Communication Technology.” Many specifically 
endorsed the creation of a new Standard under 
the DDA.” 
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Three common concerns emerged: 


1. people with disability bear a heavy burden 
in enforcing disability discrimination law by 
raising complaints under the DDA 


2. there are no legally-enforceable national 
standards for accessible technology 


3. international standards for accessible 
technology, which could benefit Australians, 
are not being applied here. 


(a) Disability discrimination law 


Some stakeholders emphasised Australia’s 
obligations under international human rights law 
regarding disability. Some, but not all, of these 
obligations have been incorporated into federal, 
state and territory laws and the National Disability 
Strategy.” 


The DDA prohibits discrimination on the 

basis of disability in employment, education, 
accommodation and in the provision of goods, 
services and facilities. An individual may complain 
to the Commission that they have suffered 
discrimination on the basis of their disability.” 


The Commission can then investigate and conciliate 
the complaint. Conciliation is an informal dispute 
resolution process that involves the complainant 
and the respondent seeking an agreed resolution 
of the dispute. The Commission is not a court and 
does not make binding decisions. 


If the dispute is not resolved through conciliation, 
the complainant can take their matter to the 
Federal Circuit Court or Federal Court of Australia, 
which can determine the matter according to 
law.’%? State and territory anti-discrimination laws 
contain similar protections and dispute resolution 
mechanisms for disability discrimination matters.®” 


Only a person who is personally affected by a 
breach of the DDA has standing to seek a remedy. 
In other words, there is no independent body to 
monitor and enforce the DDA. Many stakeholders 
reported that this creates a heavy burden on 

the disability sector, with individuals bearing 

the burden of resolving disability discrimination 
problems, including where those problems are 
systemic or affect others as well. 


There can be a power imbalance in any litigation 
process, including under the DDA. Typically, an 
individual complainant has fewer resources than 
an organisation responding to the complaint. 

It has been observed that, in practice, only 
people with structural, relational, financial and 
community support are well equipped to engage 
in formal dispute resolution such as litigation.” 
If a complaint of disability discrimination can 

be resolved in a no-cost informal complaints 
jurisdiction, such as the Commission, this can 
reduce this power imbalance. 


Outcomes of conciliation are usually confidential 
between the parties. This has significant practical 
impacts: 


e The conciliation process is designed to 
resolve the problem for the individual 
complainant, with any changes in practice 
focused on the parties to the dispute. 
However, the conciliation process is not 
directed towards systemic problems that 
extend beyond the disputing parties. In 
other words, a successful conciliation 
can result in practical changes at the 
organisation that was subject to the 
complaint, but not at other companies in 
the same sector where similar problems 
may also exist. 


The conciliation process does not produce 
legal precedent or judicial guidance on what 
is required to ensure equal treatment and 
avoid discrimination. While the Commission 
provides de-identified information 

about the resolution of some significant 
complaints, this is necessarily limited. 


The emotional, social and financial costs to 
complainants under the DDA can have especially 
adverse impacts on people with disability.°” 
Advocacy fatigue is common among people who 
are already exposed to systemic inequalities and 
who are continually fighting for their rights.8° The 
University of Technology Sydney observed that 
‘large cases are rare, and they expose a plaintiff to 
considerable financial risk’.8% 


Digital Gap Initiative submitted: 


[F]rom first-hand experience, the burden imposed 
on individuals with disabilities to seek legal 
remedies under Australian law is far too onerous. 
Placing the onus on minority communities to 
lodge complaints and fight for change presents an 
unnecessary challenge.®°> 


The Commission is currently finalising 
recommendations for reform of federal 
discrimination law, including the DDA, as part of 
the Free and Equal project. A report, to be released 
in 2021, will propose ways of improving access 

to justice, how to create a proactive, preventative 
culture, as well as modernising Australia’s 
discrimination laws. 


(b) An obligation to provide accessible Digital 
Communication Technology 


As noted above, community and advocacy groups 
expressed support for a new Standard under the 
DDA.®° This would be in addition to the current 
DDA Standards, which cover the areas of education, 
buildings and public transport.®°7 


Stakeholders submitted that a new standard 

under the DDA would help relieve the burden on 
the disability sector to redress inaccessible Digital 
Communication Technology. Kingsford Legal Centre 
submitted: 


The reactive nature of the complaints system 
means that it is difficult to address inaccessible 
technologies once they have already been made 
available to the market.8°8 


These stakeholders observed that guidelines, such 
as Web Content Accessibility Guidelines (WCAG) 2.1, 
are not legally enforceable. Standards Australia is 
not a government regulator and does not enforce 
or monitor compliance with standards. Some 
stakeholders suggested that the voluntary nature 
of such standards presents more of a problem if 
the law does not provide sufficient protection. For 
example, the Digital Gap Initiative stated: 


While voluntary standards are a positive start, 
no meaningful action will be taken without some 
formal mandate and even punitive measures for 
businesses which fail to comply with legislative 
requirements.®'° 


The Digital Gap Initiative concluded: 


In our view the DDA should be the focal point for 
standards and enforceability of accessibility. At the 
end of the day, disability discrimination is against 
the law and this should be emphasised and 
leveraged where possible.®"! 


Blind Citizens Australia urged 


the promotion of mandatory and implementable 
standards in this space in order to safeguard 

the rights of Australians who are blind or vision 
impaired to full and equal access to technology, 
which is critical to our ability to contribute to 

the social, economic and cultural life of our 
communities.®"? 


Stakeholders suggested a range of approaches and 
features to support the success of the proposed 
Standard, including that it should:°'? 


adopt best practice from existing 
international standards 


be developed in consultation with people 
with disability and the technology industry 


be reviewed regularly to address swift 
technological change 


include a range of indicators to measure 
accessibility and compliance, and 
incorporate ‘human rights by design’ and 
universal design principles, notably co- 
design with people with disability®"4 


include civil penalties for non-compliance 
and for vendors who provide misleading 
information about their accessibility 
compliance. 


Several stakeholders submitted that the success 
of a Digital Communication Technology Standard 
would be enhanced with strong, independent 
regulatory oversight of its operation. This could 
help support the implementation of Australia’s 
international treaty obligations and complement 
any industry self-regulatory measures.®'° 
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Some stakeholders anticipated the potential 
significant impacts of a new legally enforceable 
standard on the technology industry. Some 
suggestions were made that would support 
industry with this transition including that: 


* the introduction of a new standard should 
involve an open, consultative approach 
between any regulatory body and 
technology service providers®'® 


the standard should adopt a progressive 
approach to implementation, to help 
achieve true compliance through building 
organisation awareness and capacity? 


resourcing and funding should be made 
available to assist organisations comply.®"® 


12.4 Improving operation of 
disability discrimination law 


(a) Experience of inaccessible Digital 
Communication Technology 


There is no comprehensive national data about the 
accessibility of goods, services and facilities that use 
Digital Communication Technology. However, the 
problem of inaccessibility is pervasive in this area, 
and it limits the rights of people with disability. 


The Commission heard from a broad range of 
stakeholders across the nation over two rounds of 
public consultation. Many of the views expressed 
are broadly consistent with stakeholder input to 

a 2017 parliamentary committee on inclusive and 
accessible communities,?"? and a 2009 report on the 
experiences of people with disability in Australia.®° 


Where Digital Communication Technologies are 
inaccessible, this can have a profoundly negative 
impact on people with disability. As a State Party 
to the CRPD, Australia should act decisively to 
eliminate barriers to accessibility. 


(b) Anew Standard under the Disability 
Discrimination Act 


The CRPD requires Australia to adopt appropriate 
measures in law for the full realisation of human 
rights for people with disability,**! and to effectively 
monitor implementation of the CRPD itself.8 The 
United Nations (UN) Committee on the Rights of 
Persons with Disabilities has recommended that 
Australia take the necessary legislative and policy 
measures, including through the use of public 
procurement policies, to implement the full range 
of accessibility obligations under the CRPD and 
ensure effective sanctions or remedies for non- 
compliance.° 


This goal can be advanced in a number of ways. 
For example, compliance and data collection 
reforms are being considered by the Commission 
in its Free and Equal Project. In 2021, the Disability 
Discrimination Commissioner will also be reviewing 
the effectiveness of standards made under the 
DDA and how compliance with standards may be 
improved. 


This Project has focused on another critical reform: 
the creation of a new Digital Communication 
Technology Standard under s 31 of the DDA. The 
DDA’s protection should be improved as it applies 
to Digital Communication Technologies, as people 
with disability encounter multiple barriers when 
accessing goods, services and facilities that rely on 
these technologies. 


(i) Benefits of a new Standard 


The development of a new Digital Communication 
Technology Standard would have a number of 
potential benefits. 


It is likely to increase availability of accessible 

Digital Communication Technology for people 

with disability, reducing the likelihood of unlawful 
discrimination. A binding standard would clarify, 

for government and the private sector, the DDA's 
minimum requirements regarding goods, services 
and facilities that use Digital Communication 
Technologies.®” It wound send a clear message that 
people with disability use Digital Communication 
Technology and so it needs to be accessible. 


Such a standard would help guide the design and 
development of those goods, services and facilities, 
thereby spurring innovation that promotes 
accessibility for people with disability. It would 

also provide legal recourse for anyone negatively 
affected by a good (such as a product), service or 
facility that fails to comply with the Standard. 


There may also be broader commercial and 
community benefits. Accessible design and 
development can give businesses a competitive 
advantage, increased market share and enhanced 
reputation, and greater adaptability to external 
forces such as market changes and regulatory 
reform. 


(ii) Addressing difficulties in implementing a 
new Standard 


A new Digital Communication Technology Standard 
cannot be a panacea. It will have limitations in its 
effectiveness, and these need to be considered 
when deciding on what is included in the new 
Standard. 


Standards are delegated legislation under the DDA 
and require individuals to bring complaints, collate 
evidence, prove facts and understand potential 
legal defences. Therefore, it may be additional 
regulation is also needed to protect the functional 
accessibility of goods, services and facilities that 
rely on Digital Communication Technology. For 
example, this may be the case where a sector 
regulator is required to protect the interests of 
individuals who might struggle to advocate on their 
own behalf. Any additional or different regulation 
should complement the proposed Standard and 
reflect Australia’s obligations under the CRPD. 


(iii) The coverage of a Digital Communication 
Technology Standard 


A Digital Communication Technology Standard 
should cover the provision of publicly-available 
goods, services and facilities that use digital 
technologies for communications purposes. This 
would include any software, hardware, VR and AR 
technologies used in: 


- ICT: for example, desktops, laptops and 
mobile devices 


- websites, as well as features and tools 
embedded in online platforms (eg, 
CAPTCHA) 


- public-facing communication platforms 
in areas such as banking (eg, ATMs, 
EFTPOS machines) and travel (eg, ticketing 
machines, kiosks). 


There was some support for including loT-enabled 
technologies in a new DDA Standard.®* Goods and 
services that use loT tend to operate via a user 
interface, which allows people to input and receive 
information. The user interface is the primary 
barrier for people with disability when accessing 
loT-enabled devices. By including within its scope 
the user interface of loT-enabled goods, services 
and facilities, the Standard would include home 
automation products, household appliances, 
wearable technology (eg, smart watches), health 
appliances and remote monitoring. 


As discussed below, the new Standard should also 
cover information that is needed to operate goods 
and services—such as instruction manuals. 


(iv) The development and content of the new 
Standard 


The process of developing a new Standard under 
the DDA will be critical to its success. 


Some in industry are already making accessible 
products and services that incorporate Digital 
Communication Technology, and they would 
require no or minimal adjustment. Others might 
need more support and capacity building. 


That could be achieved through a range of 
measures. For example, Chapter 15 explores 
education and professional development to support 
industry understand and implement accessible 
practices and policies. In addition, a new Standard 
under the DDA could provide for progressive 
compliance milestones, as has been the experience 
with some existing Standards.®¢ This would give 
industry time to adjust its design, development 
and manufacturing processes to comply with the 
requirements of the Standard. 
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There should be open consultation between the 
Australian Government, technology industry 

and the community in developing the Standard. 
This will be aided by the Disability Discrimination 
Commissioner's forthcoming review of standards 
under the DDA. In addition, the consultation 
process should consider the following factors: 


* A balance needs to be struck between 
principles, outcomes and rules-based 
content to allow flexibility for different 
contexts, and yet be specific enough to be 
useful for technology developers. Some 
outcomes-based elements could help 
balance the need to promote innovation 
with providing regulatory certainty. 
Principles of ‘human rights by design’ 
could help industry incorporate human 
rights norms into product design (through, 
for example, co-design with people with 
disability).°?” 


Standards should adopt international best 
practice guidelines and allow for a range of 
other indicators to measure success, such 
as inclusion of innovative and co-design 
practices. Compliance measures should 
cover existing goods, services and facilities 
that are inaccessible, and allow for gradual 
and systematic removal of barriers under 
continuous monitoring, with the aim of 
achieving full compliance.®8 


Standards need to be updated periodically 
to account for technological changes, 

and also provide a level of certainty for 
developers as they manufacture and 


release products according to the Standard. 


(c) Compliance framework and data 
collection 


There is currently no comprehensive mechanism 
for monitoring and enforcing compliance with 

the DDA. This can contribute to the financial and 
emotional burdens on individuals who make a 
complaint under the DDA, especially if a complaint 
cannot be resolved informally or by conciliation.8° 
There has been difficulty in ensuring that rights are 
upheld with respect to accessibility for people with 
disability in the absence of additional enforcement 
mechanisms—something recognised by the UN in 
its assessment of Australia’s implementation of the 
CRPD.8 


The DDA contains measures to promote compliance 
and enable remedies for people affected by 
non-compliance. These measures, including the 
existing Disability Standards, have had only mixed 
success in achieving this goal.83' On one hand, the 
current Standards have helped to clarify general 
legal principles. However, they have not always 
been practical and effective in sanctioning non- 
compliance.832 


Consequently, in addition to the creation of 

the proposed new Digital Communication 
Technology Standard, broader reform may be 
necessary to ensure Australian laws dealing with 
disability discrimination effectively fulfil the CRPD 
obligations. Some of these larger questions of 
discrimination law reform are being considered 
separately. 


In particular, the Commission's Free and Equal 
Project has proposed a modernisation of the 
regulatory framework to include more effective 
enforcement powers for the Commission. These 
might include audits; own-motion investigations; 
compliance notices; enforceable undertakings; and 
applications to the Court to require compliance. 
The Commission, in that Project, proposes building 
and supporting a culture of prevention so that duty- 
holders take responsibility to ensure discrimination 
does not occur. 


Similarly, in his comprehensive review of 
DDA standards, the Disability Discrimination 


12.5 Government services and 
activities 


Commissioner is considering ways to achieve: more 
effective monitoring of compliance with standards; 
higher rates of compliance; greater clarity about 
legal obligations; and a more accessible and 
effective complaint-handling process for the 

most vulnerable people in the community.°4 The 
Commission acknowledges there is scope for a 
greater emphasis on systemic discrimination and 
placing less pressure on individual complainants, 
as well as a review of the financial costs associated 
with court cases where the complaint is not 
resolved at the Commission level. 


In the Free and Equal Project, the Commission 
proposes enhancing access to justice for 
complainants and improving the practical operation 
of anti-discrimination law. Specific reforms could 
include: shifting the evidentiary burden at the 
judicial phase, after the establishment of a prima 
facie case; limiting the costs liabilities of meritorious 
complainants, subject to established criteria; 
allowing unions and representative groups to bring 
a claim; and reviewing funding for community legal 
centres and legal aid to ensure adequate resources. 


The Digital Communication Technology Standard 

is more likely to achieve its aims if these kinds of 
additional measures can strengthen the legislative 
framework, support providers of goods, services 
and facilities to comply with their obligations, and 
help empower members of the community to make 
a complaint if needed. 


RECOMMENDATION 25: The Australian 
Government and state, territory and 

local governments should commit to 
using Digital Communication Technology 
that fully complies with recognised 
accessibility standards—especially WCAG 
2.1 and Australian Standard EN 301 549, 
and successor standards. To this end, all 
Australian governments should: 


(a) introduce whole-of-government 
requirements for compliance with 
these standards, including by: 


* providing information that is 
publicly available about how 
each agency complies with 
these requirements, reported 
annually 


establishing central line agency 
and ministerial responsibility for 
monitoring compliance across 
government 


resourcing training and advisory 
support to assist compliance 


(b) promote accessible goods, 
services and facilities that use Digital 
Communication Technology by 
favouring procurement from entities 
that implement such accessibility 
standards in their own activities 


(c) develop policies and targets 

to increase the availability of 
government communications in Easy 
English and provide human customer 
supports for people with disability 
who need to communicate with 
people instead of accessing digital 
services. 
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This section considers the role of all levels of 
Australian government in adopting and promoting 
accessible Digital Communication Technology. 


The Commission recommends the Australian 
Government adopt a whole-of-government 
approach to the procurement and use of accessible 
goods, services and facilities that involve Digital 
Communication Technology. A key goal should be 
to enhance compliance with the latest international 
accessibility standards. 


(a) Experience of government services 


People with disability have varied experiences when 
communicating with government agencies. Some 
reported positive examples of accessible services 
and facilities. Others reported inaccessible 
communication with agencies across all levels of 
government.® 


There were several suggestions for improving 
government policy and practice. First and most 
importantly, it was suggested that there should 

be better adherence to WCAG 2.1 and Australian 
Standard EN 301 549.87 Stakeholders observed that 
the digitisation of government services has made 

it difficult for people with intellectual disability and 
require human customer supports.88 One solution 
would be to provide important government 
information in Easy English.®9 


The Australian Government adopted Australian 
Standard EN 301 549 in 2016.8 This deals with 
accessibility requirements suitable for public 
procurement of ICT products and services.*" 


The Australian Government also encourages 

its agencies to meet WCAG 2.1 in all 
communications." WCAG 2.1 aims to provide a 
single, shared accessible standard for web content 
to a wide range of people with disability including: 
blindness and low vision; deafness and hearing 
loss; learning disabilities; cognitive limitations; 
limited movement; speech disabilities; and 
photosensitivity. 


The Australian Government's National Transition 
Strategy, to implement the earlier WCAG 2.0 
version, commenced in 2010 with the aim of 
implementation across all levels of government 


over four years.“ There is no recent Government 
audit or evaluation of that strategy and adoption 
of WCAG 2.0.35 An external assessment suggested 
that conformity with WCAG 2.0 varied across 
agencies, and that the strategy was ‘successful 

in the raising of awareness of the issues and 
requirements of website accessibility, particularly 
for government agencies’.®“° 


(b) Government procurement 


Stakeholders across civil society, academia and 
industry supported all levels of government being 
required to procure accessible ICT.°4” 


The effectiveness of any government procurement 
policy turns on the extent to which it is followed. 
Some stakeholders observed that previously 

such policies were not consistently applied across 
government, and suggested that new monitoring 
and compliance measures be introduced to support 
implementation of new procurement policies. 


Stakeholders advocated that government 
procurement rules incorporate some or all of the 
following accessibility components: 


* application to government agencies, 
vendors, contractors and partners, as well 
as essential industries used by the public 
such as finance, healthcare and legal 
organisations®° 


compliance mechanisms allowing 
government employees to sue their 
employer agency for failing to provide 
accessible equipment required to do their 
jobs? 


compliance measures to test accessibility, 
such as EN 301 549 compliance 
documentation or Voluntary Product 
Accessibility Templates,5 and allow for 
progressive realisation as accessibility 
becomes better understood and practiced 
across government services®*2 


advantages for vendors who comply 
with accessibility standards in their own 
operations®*? and promote inclusive 
design.®4 


(c) Conclusion 


Some government agencies are already taking 
positive steps to promote accessibility.°°° However, 
the experience of people with disability accessing 
public services varies and depends on the service 
and the person's disability. 


The CRPD includes a requirement for States Parties 
to provide accessible public services,2°° which 
necessarily entails the procurement of accessible 
goods, services and facilities. The UN Committee 
on the Rights of Persons with Disabilities noted 
that ‘it is unacceptable to use public funds to 
create or perpetuate the inequality that inevitably 
results from inaccessible services and facilities’.857 
In applying this requirement to Australia, the 
Committee found that Australia should develop 
public procurement criteria to implement the full 
range of CRPD accessibility obligations for ICT.858 


Government leadership is vital in promoting 
human rights compliant Digital Communication 
Technology, especially through the provision of 
public services. Government procurement policies 
can influence markets and be a lever for change in 
the public and private sectors.%? 


For example, the Commonwealth Indigenous 
Procurement Policy incentivises the inclusion of 
Indigenous enterprises at all levels of the supply 
chain to ‘help maximise the development of the 
Indigenous business sector’.8°° Overall, the policy 
has had a positive effect on Indigenous business.**' 
Since 2015, the policy has successfully generated 
over $3.5 billion in contracting for Indigenous 
businesses, across 24,470 contracts awarded to 
2,140 businesses.° 


Outside government, procurement policies are also 
gaining traction, as demonstrated by the Business 
Council of Australia’s ‘Raising the Bar’ initiative 
which is a $3 billion procurement agreement 
between some of its member organisations and 
Indigenous suppliers.°° 


The Commission recommends that the Australian 
Government adopt a whole-of-government 
approach to the procurement and use of accessible 
goods, services and facilities that involve Digital 
Communication Technology. A key goal should be 
to enhance compliance with the latest international 
accessibility standards. 


The Commission recommends that accessibility 
requirements be included in the Government's 
procurement frameworks, such as the 
Commonwealth Procurement Rules, as well as 
in corresponding state and territory procurement 
rules and policies. This would have a number of 
benefits. 


e It would enhance accessibility for public 
sector employees with disability. Inclusive 
and accessible workplaces help attract 
and retain employees with disability, as 
they are more likely to feel supported and 
encouraged in their employment.®© 


It would enhance accessibility for the 4.4 
million (about 1 in 5) Australians with 
disability when they interact with public 
services.°°” 


It would signal to the private sector the 
importance of manufacturing accessible 
Digital Communication Technology and 
incentivise meeting these requirements 
for any involvement in public sector 
procurement or funding. 


This would, in turn, improve accessibility 
across large public and private institutions 
that procure similar products and services 
(for example, universities, hospitals and 
banks). An increase in demand from the 
public service for accessible goods and 
services would likely result in increased 
availability and competition across the 
market. 


As the market shifts more towards 
accessible goods and services, it would 
promote best practice and showcase 
leading providers. 


The Australian Government should consider 
several guiding principles in the development and 
implementation of accessibility requirements in 
procurement at the federal level:868 


e people with disability and their 
representatives should be involved in the 
development, testing and evaluation of 
accessible services 
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* the government should commit to building 
the capacity of industry to respond to this 
requirement through early and frequent 
engagement, and support the involvement 
of small and medium enterprises 


the government should monitor and 
evaluate policies to ensure consistent 
implementation and effectiveness across 
government 


* the government should promote innovative 
public procurement processes through 
activities such as technology contests 


government agencies and departments 
should collaborate on the implementation, 
successes and challenges of the policy, to 
improve outcomes and learn from each 
other. 


12.6 Improving private sector 
use of accessible Digital 
Communication Technology 


RECOMMENDATION 26: The Australian 
Government Department of Industry, 
Science, Energy and Resources or the 
Digital Transformation Agency should 
conduct an inquiry into compliance by 
industry with accessibility standards such 
as WCAG 2.1 and Australian Standard EN 
301 549. 


The inquiry should consider the extent 

to which incentives for compliance with 
standards should include changes relating 
to taxation, grants and procurement, 
research and design, and the promotion 
of good practices by industry. 


In addition to law reform, it was clear from the 
Commission's consultation that the public and 
private sectors could benefit also from better policy 
and self-regulation in this area. 


There is limited hard data on private sector 
compliance with accessibility requirements in 
respect of Digital Communication Technologies. 
This data is needed to formulate a plan for how 
government could most effectively incentivise good 
practice in the private sector. 


Civil society and industry stakeholders supported 

a government inquiry into compliance by industry 
with accessibility standards.86° Several focused on 
the role that government incentives might play in 

lifting industry performance, such as: 


* incorporating accessibility requirements 
into the terms of government grants®”° 


tax concessions and grant incentives 
to businesses that provide accessible 
technology?” 


a trust mark to symbolise a business’ 
adherence to accessible technology,®? or 
a requirement that Australian businesses 
make information about the accessibility 
of their goods and services available and 
accessible to consumers®? 


industry awards, prizes and showcases for 
best designs and processes.°” 


Telstra suggested any audit into accessibility 
standards compliance by industry be conducted 
progressively: 


There is currently no real national standard in this 
space and disability advocates tell us compliance 
with the different standards that do exist is 

poor. We suggest a progressive introduction 

of standards, with audits conducted after the 
standards have been introduced, and that the 
technology sector must be involved in designing 
and testing the standards to ensure they are 
workable.®”° 


Conclusion 


The Australian Government should stimulate 
the development and use of accessible Digital 
Communication Technology through policies 
that favour industry participants that promote 
accessibility. 


Stakeholders provided examples of the kind of 
incentives that could promote accessibility across 
industry, including tax concessions to accessible 
businesses, industry awards, accessible trust marks, 
and prescribing accessibility in government grants. 


In addition to promoting compliance with 
accessibility standards WCAG 2.1 and AS EN 301 
549, government incentives such as grants and 
research opportunities might be provided for 
industry to develop and adopt innovative ways of 
communicating with people with disability. 


The Commission heard from industry stakeholders 
that want to adopt more accessible practices, and 
from accessibility consultants and disability groups 
who have collaborated with business to achieve 
positive accessibility outcomes. There are clear 
opportunities for further improvements across the 
private sector.8”6 


Collaboration across industry and non-government 
organisations has a role to play in promoting 
accessible technology. For example, with funding 
from ACCAN, the Melbourne Social Equity Institute 
developed a program in collaboration with industry, 
people with disability and mental health consumers 
to support providers of goods and services with 
their communications with people who have a 
cognitive disability.°”” 


However, there is limited data on industry 
compliance with accessibility standards. In the 
Commission's view, an inquiry is needed to expose 
barriers to the adoption of accessibility standards, 
as well as reveal good practices and innovation in 
the area. The Digital Transformation Agency (DTA) 
and the Department of Industry, Science, Energy 
and Resources (DISER) would each be well placed to 
lead such an inquiry. 


DISER has a Strategic purpose to improve business 
capability and support ‘industry transformation by 
encouraging innovation and investment’.®” The DTA 
has a national role in leading and shaping whole- 
of-government policies on shared ICT and digital 
services.®”? The DTA supports government agencies 
in adopting accessible services through its Digital 
Service Standard.®®° 


Together, the DTA’s expertise with understanding 
barriers to the provision of accessible services, 
and DISER’s business development and innovation 
focus, could lead this inquiry and help identify 
incentives to improve accessible policies and 
practices. 
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13.1 Summary 


News, information and entertainment are central to democratic and cultural life. 

Until recently, broadcast television was the dominant way of watching this material. 
Technological developments, which have accelerated in the 21% century, have resulted in 
an ever-increasing range of media that provide this content to us—including free-to-air 
broadcasting, subscription television, video and online platforms. 


The use of accessibility features—especially captioning and audio description—can 
enable this audio-visual content to be enjoyed by people with disability, and especially 
people with disabilities affecting their vision or hearing. As the media and platforms on 
which audio-visual content increase and fragment, ensuring accessibility of this content 
has become more complex. 


Many people with disability experience difficulty in obtaining functionally accessible 
audio-visual news, information and entertainment. Stakeholders focused especially on 
the level of unmet need for captioning and Auslan interpretation for people who are deaf 
or who have a hearing impairment; and audio description for people who are blind or 
who have a vision impairment. 


People with disability have a right to access news, information and entertainment." 

The Commission considers that this right should apply to all the principal ways in which 
people access such content in the 21* century, including broadcasting services on free- 
to-air and subscription television, as well as newer forms of content transmission such as 
online streaming services and social media. 


This chapter includes recommendations to address this problem of functional 
accessibility by facilitating: 


1. increased audio description and captioning requirements for broadcasting 
services, as well as video, film and online platforms 


2. improved provision of accessible information during emergency and important 
public announcements 


3. better monitoring of compliance with accessibility requirements and voluntary 
targets for the distribution of audio-visual content. 
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13.2 Accessibility of traditional 
broadcasting services 


RECOMMENDATION 27: The 

Australian Government should amend 
the Broadcasting Services Act 1992 (Cth) to 
increase the amount of accessible content 
available for people who have hearing or 
vision difficulties as follows: 


(a) national and commercial free- 
to-air television services should be 
required to provide audio described 
content for a minimum of 14 hours of 
programming per week, distributed 
across the primary and secondary 
channels. This should be increased 
to a minimum of 21 hours per week 
in a timeframe to be determined 

in consultation with people with 
disability and broadcasting services. 


(b) subscription television services 
should be required to provide audio 
described content for a minimum 


of 14 hours of programming per 
week for their main channels. This 
should be increased to a minimum 
of 21 hours per week in a timeframe 
to be determined in consultation 
with people with disability and 
broadcasting services. 


(c) national and commercial television 
free-to-air services should be 
required to increase the captioning 
of their content on an annual basis, 
resulting in all such broadcasting 
being captioned on primary and 
secondary channels within five 
years. The Government should 
determine a formula for annual 
progressive increases of captioning 
in consultation with industry, 
people with disability and their 
representatives. 


RECOMMENDATION 28: The 
Australian Government Department 
of Infrastructure, Transport, Regional 
Development and Communications 


should conduct a review to identify 
effective, practical ways to increase audio 
description and captioning on secondary 
or specialist broadcast television 
channels. 


People with disability have a right to access audio- 
visual content, regardless of how that content is 
distributed. Australian law should recognise and 
protect this right. 


Captioning has become more cost effective 
through advances in technology and as policies, 
practices and regulation have made captioning 
more common. Although audio description was 
developed more recently than captioning, the 
cost associated with audio description are also 
decreasing over time. Nevertheless, the obligation 
to provide accessible services will require industry 
commitment and dedicated resources. To achieve 
this goal, some broadcasters may need to re- 
prioritise their functions and activities; others may 
need some Government support. 
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AD)) 


(a) Audio description 


In the Discussion Paper, the Commission proposed 
that broadcasters be required to audio describe 

a minimum of 14 hours of content per week with 
annual increases, and to increase current minimum 
hours of captioned content.®° 


Disability and other civil society groups 
overwhelmingly supported a legislated requirement 
for free-to-air and subscription television channels 
to provide a minimum of 14 hours of audio 
description per week, with minimum annual 
increases.*®? This is consistent with broader 
community advocacy for mandatory audio 
description quotas.®%4 


The ABC and SBS started broadcasting on average 
approximately 14 hours of audio-described 
programming per week in June 2020.8% The 
national broadcasters each received $2 million 
from the Federal Government to fund their audio 
description services in 2020-21.°° Prior to this, the 
Government funded trials of audio description on 
ABC1 in 2012 and on the ABC's online streaming 
platform, iview, in 2015-16.°°” 


The Commission's final round of community 
consultation took place at around the same time 

as the new Government-funded audio description 
services started. While there was support for 
increased audio description in broadcasting, many 
people with disability described feeling uncertain 
and disappointed with the changing policy positions 
in Australia, despite long-term advocacy to effect 
change. 


The 2015-16 audio description trial in Australia did 
not lead to significant change, despite its apparent 
success, prompting one consultation participant to 
say: 


| don’t want to get my hopes up. | started to get 
into some TV series the last time around and 
stopped watching TV after the last trial ended. It’s 
too psychologically difficult and devastating. This 
[audio description] is not just a nice-to-have.*® 


One disability group representative spoke about 
their membership base being 


concerned about what will happen after the 
funding finishes in 12 months. They've been 
guinea pigs before for trials and they're worried 
they're going to lose [audio description] again. 
[Audio description] can't keep being given, and 
then taken away.®%° 


Some disability representative groups submitted 
that the most important feature of any new policy 
is certainty that it will be adopted and enforced. 
They suggested audio description is most likely 

to succeed in Australia if industry is supported by 
government with the establishment of necessary 
infrastructure needed for audio description.®" 


It was submitted that a graduated approach to 
audio description—a minimum number of 14 
hours per week and annual increases—would help 
ensure industry commitment to audio description 
in Australia.®°? Representative groups noted that 
the most effective way to provide certainty for the 
community and ensure long-term implementation 
of audio description is to legislate minimum 
quotas in the Broadcasting Services Act 1992 (Cth) 
(Broadcasting Services Act).°93 


Some advocates noted that now is an opportune 
moment for the Australian Government to legislate 
for audio description, and to act on the culmination 
of concerted advocacy efforts over decades, as 

well as current ABC and SBS audio description 
services. 


Stakeholders also urged that, in addition to 
broadcasting, audio description quotas be 
introduced for online content, video content, and 
streaming services.®° 


(b) Captioning and Auslan 


Several disability representative groups urged an 
increase in captioned content quotas on broadcast 
television, with improved measures for quality 
assurance.®* Currently, free-to-air broadcasters 
are required to caption 100 per cent of programs 
broadcast on main channels between 6am 

and midnight, and all news and current affairs 
programs on main channels, regardless of the 
time at which they are broadcast.” This basic 

rule does not extend to television broadcasters’ 
secondary channels (sometimes known as ‘multi- 
channels’) unless the program has been broadcast 
with captioning on another of the broadcaster's 
television services.°%8 


ACCAN recommended an increase to captioning 
quotas: 


We are keen to ensure that this requirement 
also applies across national, commercial and 
subscription broadcasting services, and across all 
channels (including multi-channels). We strongly 
believe that broadcasters should be providing 
captioning for the full 24 hour viewing day. 
Furthermore, efforts must be made to ensure 
that access features follow video content across 
different platforms.®°? 


Disability representatives noted that accessibility is 
compromised for people who are deaf or hearing 
impaired when captioning does not accurately 
represent the content. This can be a problem when 
captioning is automated and not checked for error 
before being transmitted to the viewer.°” 


(c) Secondary and specialist broadcast 
channels 


Part of the rationale in imposing a limited quota for 
audio described content is a recognition that some 
programming is more difficult to audio describe. 
People who use audio description have previously 
reported that scripted content (such as drama and 
documentary) is better suited to audio description, 
as compared with live or near-live content (for 
example, news and live sports).°" 
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Moreover, following reform to Australian 
broadcasting law since 2000, which was coupled 
with the introduction of digital broadcasting, 
television networks are now permitted to have a 
primary channel, as well as one or more secondary 
or ‘multi’ channels (referred to in this Final Report 
as ‘secondary channels’). 


The popularity of secondary channels has increased 
since their introduction. However, commercial and 
government-owned television networks sometimes 
offer more specialised content on secondary 
channels, such as dedicated foreign language, 
music, sports and news channels. This can appeal 
to, or be suitable for, a smaller number of people 

in the community, as compared with the content 
typically broadcast on primary channels. In addition, 
sometimes secondary channels include material 
that is re-broadcast directly from international 
providers. This combination of factors can make the 
content on these secondary channels less suited to 
audio description, for cost and practical reasons. 


(d) Conclusion: a need for reform 


The Commission recommends reform to law and 
policy to ensure that people with disability can 
enjoy the right to access audio-visual content, 
regardless of how that content is distributed. 


The changes recommended by the Commission will 
require broad consultation. Consistently with the 
requirements of the CRPD, that consultation should 
include people with disability.°°* As ACCAN has 
observed, such consultation should 


occur throughout the reform process. People 
with disability must be kept informed about 
changes that will affect them and must have the 
opportunity to have their say regarding these 
changes.° 


(i) Audio description 


The ABC and SBS provide an average of 14 hours 
per week of audio description to Australian 
audiences. The ABC and SBS spread these 14 hours 
of audio described content across their primary 
and secondary channels.*™ There is further, but 
limited, availability of audio description through 
some DVDs, specialist cinema screenings and via 
international subscription video-on-demand (SVOD) 
services. %°> 


Audio description is primarily used by people who 
are blind or have a vision impairment. In addition, 
some sighted members of the community also 
report using audio description. For example, 

some people with autism spectrum disorder and 
intellectual disability use audio description to 
better enjoy and understand the content, and 
some other people find it useful when multitasking 
or attempting to understand culturally-specific 
references.” 


Australia is the only English-speaking OECD country 
without compulsory minimum quotas for audio 
description on free-to-air television.” Free-to-air 
broadcast services are the most popular medium 
for older people, who make up almost three 
quarters of people who are blind or vision impaired 
in Australia.°% This proportion is predicted to 
increase with an ageing population and the age- 
related nature of vision loss.°° 


People who are blind or have a vision impairment 
have long advocated minimum quotas for audio 
description on free-to-air television, video-on- 
demand and subscription broadcasting.*"° Senator 
Steele-John introduced a private member's bill in 
2019, proposing a minimum of 14 hours per week 
per channel of audio description for the first three 
years of operation, followed by annual increases to 
21 and 28 hours per week.?"! 


In December 2017, a working group of the 

then Australian Government Department of 
Communications stated that ‘significant work would 
be required on the broadcast transmission side and 
the consumer receiver side before an AD service 
could be introduced’.?"* Free-to-air broadcasters 
expressed concerns at the time that if audio 
description were mandated in law, this would cause 
structural, technical and financial disruptions to 
their services.?'3? The Commission acknowledges 

the significant effort needed for free-to-air 
broadcasters to provide audio description services 
to the Australian public. 


The ABC and SBS consulted people with disability 
and their representative groups to prepare for the 
implementation of their audio description services. 
They consulted on suitable audio description 
formats that would align with TVs and set top 
boxes, as well as audience preferences for audio- 
described programming and information regarding 
the service. Since its inception in mid-2020, the 
broadcasters have received positive feedback from 
the community and are continuing to engage with 
them throughout the process.?"* 


ABC's and SBS's groundwork to address some 
of the technical challenges and consult with 
the community could help other broadcasters 
implement audio description services. 


The Commission supports calls for broadcasting 
content to be made more accessible for people with 
disability. As a first and important step, adopting 

a minimum audio description requirement of 

14 hours per week for national, commercial and 
subscription broadcasting services would benefit 
many Australians.?'° 


This minimum quota of 14 hours per week should 
be increased to 21 hours in an appropriate 
timeframe, to be determined in consultation with 
people with disability and broadcasting services. 
The Australian Government should also provide 
reasonable support to national, commercial and 
subscription broadcasting services to help with 
the establishment of these services. Over time, 
minimum quotas should continue to be increased 
further. 


(ii) Captioning 


Under the Broadcasting Services Act, broadcasters 
on free-to-air television currently must provide 
captioning for: 


* content broadcast between 6am and 
midnight on primary channels 

* news and current affairs programs at any 
time on primary channels 


* any program that is broadcast as a ‘repeat’ 
on secondary channels having previously 
been broadcast on a primary channel. 


FIRE INFORMATION LINE 
1800 NSW RFS 
1800 679 7 
www. rfs.nsw.g 
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Stakeholders representing people with disability 
generally urged that the Act be amended to 
require captioning for all content on free-to- 

air television 24 hours per day. This position 

is consistent with the views described in the 
Australian Communications and Media Authority 
(ACMA) report of 2017 on captioning obligations 
in the Broadcasting Services Act.?'° Representative 
groups have previously advocated the extension 
of minimum hours of captioned content across 
free-to-air secondary or multi-channels, rather 
than applying only to a broadcaster's primary 
channel.” In that review, commercial and national 
broadcasters opposed an extension to 24 hour 
captioning, citing concerns regarding additional 
costs.°'8 


People who are deaf or have a hearing impairment 
have a right to access broadcasting services, 
including news and entertainment broadcasting. 
The Commission recommends that the minimum 
requirement for captioned content on free-to-air 
broadcasting be increased to support this right. 
Progressive quotas and targets should help national 
and commercial broadcasters gradually build on 
their existing systems and processes to achieve this. 


Subscription broadcasting services are already 
subject to legislated annual increases of content 
that must be captioned.?'? There are nine categories 
of subscription television. Each category has a 
captioning target, which increases by 5 per cent 
each year until it reaches 100 per cent. In 2021, all 
the content on some dedicated movie channels 
and 90 per cent on some dedicated entertainment 
channels must be captioned. News and sports 
currently have a target of 55 per cent, which will 
reach 100 per cent in 2031.97? The Commission 
acknowledges that this already provides a pathway 
towards captioning of all content for subscription 
television. 


(iii) Secondary and specialist broadcast 
channels 


There are commercial and other factors that free- 
to-air and subscription television service providers 
consider in offering content. 


For free-to-air channels, the difference in average 
audience numbers between primary and secondary 
channels can be significant, but often less so 

than for subscription television. Each free-to-air 
television network currently has a primary channel 
and up to four multi-channels which contain a mix 
of general and specialised content.%" 


By contrast, the total audience for subscription 
television is spread over a much larger number 

of channels than those used by each free-to- 

air network. There are significant differences 
between subscription television channels that 
attract a relatively large proportion of the audience 
(for example, some movies and entertainment 
channels), and the dozens of other specialised 
channels that attract smaller audiences.??? 


The combination of smaller audiences and more 
specialised content can present commercial 
challenges for operators of that second tier of 
channels in complying with regulation, including 
rules related to accessibility of broadcast content. 
While policy in this area should pay due regard to 
this commercial reality, accessibility is a legally- 
protected right. Catering for the needs of people 
with disability should never be abandoned simply 
because, at times, there are costs associated with 
doing so. 


As this area of broadcasting has developed 
significantly in recent years, and it is continuing 

to develop, detailed expert analysis is needed to 
ensure practical and effective ways of providing 
accessible content—especially using captioning 

and audio description—via the second tier of 
channels on subscription television. To this end, the 
Commission recommends that the Department of 
Infrastructure, Transport, Regional Development 
and Communications undertake a review to identify 
ways to increase audio description and captioning 
on that tier of subscription television channels. 


There is diversity in viewers and reach across 
national, commercial and subscription 
broadcasting. The Department of Infrastructure, 
Transport, Regional Development and 
Communications should determine a practical 
basis of distinguishing between the two tiers of 
subscription channel broadcasting, by reference to 
factors including audience numbers and the nature 
of the content offered. 


13.3 Newer ways of distributing 
audio-visual content 


RECOMMENDATION 29: The Australian 
Government should introduce legislation 
to provide minimum requirements for 
audio description and captioning in 
respect of audio-visual content delivered 
through subscription video-on-demand, 


social media and other services that are 
not covered by the Broadcasting Services 
Act 1992 (Cth). Obligations should be 
determined in consultation with industry, 
and people with disability and their 
representatives. 


Current federal legislation deals with audio 
description and captioning on ‘traditional’ television 
broadcasting services. The legislation does not 
apply to newer or more specialised ways of 
communicating audio-visual content, such as SVOD, 
DVDs, films, social media, and online services. 


In particular, leading SVOD services such as Netflix, 
Stan, Amazon Prime and Apple TV, as well as social 
media platforms such as Facebook, Twitter and 
YouTube, are rapidly increasing in size and carrying 
an increasing proportion of content consumed 

by the Australian community at large. Hence, any 
accessibility problems with the content on those 
services takes on greater significance. 


The Commission considers that the Broadcasting 
Services Act should be amended to provide for 
minimum accessibility requirements for audio 
description and captioning on those services. 


Stakeholders expressed concern about this 
phenomenon.°”? This input was consistent with 
other civil society advocacy to increase the 
availability and promotion of audio description 
and captioning across SVOD and other content 
distribution platforms.°”4 


Many of the leading SVOD services state on their 
respective websites which programming and other 
material offer the option of audio description and/ 
or captioning, and some civil society groups provide 
lists and overviews of audio-described content. 


Some cinemas provide for audio description 

and open and closed captions for select movie 
sessions.”¢ Vision Australia provides information 
about audio-described video content at cinemas.°” 


Conclusion: new forms of content distribution 
and accessibility 


Avery significant number of Australians are using 
SVOD, online and social media services to consume 
audio-visual content, and the market share of these 
newer services is increasingly rapidly as compared 
with more traditional television broadcasting 
services. The Commission is concerned that, unlike 
those broadcasting services, these increasingly 
important services are not required to provide 
minimum accessible content. 


The Australian Government should introduce 
legislation to provide for minimum accessibility 
requirements for audio description and captioning 
on those newer services. This would have two 
principal benefits. 


First and most importantly, such reform would 
improve the human rights protection for people 
with disability who rely on audio description or 
captioning to experience audio-visual content. 
Secondly, it would provide for a fairer market 
among companies that distribute content, by 
harmonising the obligation to adhere to these 
accessibility requirements. This is consistent 

with one of the core objectives in the Australian 
Government's Media Reform Green Paper—namely, 
to reduce the regulatory imbalance between free- 
to-air television and internet-based competitors.?”° 


The Commission acknowledges that this would be 
a significant reform, and the precise detail would 
require consultation and coordination with key 
stakeholders, including people with disability, 
media and communications policy makers and 
regulators, as well as the companies involved in the 
various forms of audio-visual content distribution. 
The Department of Infrastructure, Transport, 
Regional Development and Communications and 
ACMA would be well placed to lead this consultation 
process. 
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13.4 Emergency and public service 
announcements 


RECOMMENDATION 30: The Australian 
Government, and state and territory 
governments, should ensure that people 
with disability can receive and understand 
emergency and other important public 
announcements, including by requiring 
government agencies to provide Auslan 
interpreters at their emergency and 
important public announcements. 


The Australian Government should 


amend the Broadcasting Services Act 

1992 (Cth) to require any television or 
other company, which broadcasts or 
re-broadcasts emergency and other 
important public announcements, to 
ensure that Auslan interpretation is 
visible on the screen at all relevant times; 
and captions are readable, accurate and 
comprehensible. 
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People with disability raised serious concerns about 
the accessibility of emergency and public service 
announcements. 


Some of these critical announcements are 
broadcast with Auslan and/or captioning. However, 
the availability and quality of those accessibility 
features are inconsistent. Sometimes there are 
significant inaccuracies in live captioning and 

no visible Auslan interpreter on the screen for 
emergency and public service announcements 
broadcast on Australian television. The Commission 
recommends reform to improve this critical 
accessibility issue. 


In light of the 2019-20 summer bushfires and the 
COVID-19 pandemic, stakeholders emphasised the 
importance of improving the overall performance 
regarding the quality and presence of captioning 
and Auslan interpreters in emergency and public 
service announcements broadcast in Australia. 
Stakeholders reported: 


* emergency briefings where there was no 
Auslan interpreter, or where the interpreter 
was not visible on the screen 


* emergency briefings for which the captions 
had an unacceptable number of errors 


* rebroadcast of announcements on digital 
platforms (for example, YouTube), where 
the content was inaccessible.%° 


Similar concerns were raised by the disability sector 
in the ACMA review of the captioning obligations 

in the Broadcasting Services Act.” In that 

review, the ACMA noted that some broadcasters 
had undertaken to include Auslan interpreters 
when they were present at an emergency 
announcement.*' These undertakings are not 
enforceable, and the ACMA stated that it would 
monitor future enquiries and complaints to assess 
whether any further regulatory intervention should 
be considered.°2 


Conclusion 


The CRPD requires States Parties to take measures 
necessary to ensure the safety of persons 

with disabilities in situations of risk, including 
humanitarian emergencies and natural disasters.” 
In response to the 2019 summer bushfires and the 
COVID-19 pandemic, Deaf Australia wrote to the 
Prime Minister saying: 


Deaf people need Auslan to access information 
so they can make informed decisions and take 
actions to protect themselves, their families and 
others around them. Text-based resources, such 
as captioning, do not have the same capacity to 
deliver accurate information as one would receive 
through Auslan.?4 


The Commission heard from community 
stakeholders who experienced severe anxiety and 
concern throughout the bushfires and pandemic 
due to inaccessible information during public 
emergency announcements. 


Emergency and public announcements are critical 
communication strategies during emergencies, 

and ‘play an important role in community safety by 
empowering people to make informed and timely 
decisions, thereby taking protective action’. The 
Royal Commission into National Natural Disaster 
Arrangements recommended that an education 
campaign on the Bushfire Warning System consider 
the needs of all Australians, and target people with 
disability.” The importance of providing accessible 
information and communications in the form of 
Auslan was noted in the Australian Health Sector 
COVID-19 emergency response plan.°?” 


The provision of accessible information during 
emergency and public announcements of national 
significance is essential to protect the rights of 
people with disability, including the rights to access 
information and the highest attainable standard of 
health. 


In addition to law reform, other measures may 
also be required to support people with disability 
in receiving and understanding information during 
emergency and important public announcements. 
For example, people with cognitive disability may 
require alternative forms of accessible content, 
which should be considered by the Australian 
Government in due course. 


13.5 Monitoring and compliance 


RECOMMENDATION 31: The Australian 
Communications and Media Authority 
should consult with broadcasters and 
introduce monitoring and compliance 
measures to support them to: 


(a) comply with accessible service 
requirements 


(b) provide quality accessible services 


(c) increase organisational capacity 
to comply with current and future 
accessible service obligations. 
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13. Broadcasting and audio-visual services 


There is no independent or regular monitoring 

of the accessibility of broadcasting services. Civil 
society stakeholders expressed concern about a 
lack of monitoring for compliance with captioning 
obligations (and any future audio description 
obligations), with a lack of data on the presence and 
quality of captioning. This places the responsibility 
on individuals to complain about shortfalls.” 


Similar concerns are also reflected in a recent 
research report by ACCAN and Curtin University.°9 
This report suggested that live caption quality is 
poor in certain circumstances, which may lead to 

a consumer not being able to follow the program. 
The report reiterated the vital role that captions 
play for people who are deaf or have a hearing 
impairment, and called for increased monitoring 
activities by ACMA and a review of the Broadcasting 
Services (Television Captioning) Standard 2013 
(Standard). 


This Standard, made under the Broadcasting 
Services Act, must be followed by free-to-air 
broadcasters when captioning programs. Under 
the Standard, captions must be readable, accurate 
and comprehensible.” Broadcasters are required 
to report on compliance with their captioning 
requirements through annual reports to ACMA,%*? 
and individuals may make a complaint about the 
absence of required captioning or about the quality 
of captioning.” 


The Australian Government and broadcasters 

have a responsibility to gather data and implement 
monitoring and compliance mechanisms for 
accessible broadcasting services (including 
captioning, audio description and Auslan 
interpretation). This responsibility arises from 
CRPD requirements that States Parties collect 
information and monitor the rights contained in 
the Convention.%° 


14. Availability of new technology 


14.1 Summary 


Accessibility includes ensuring that goods, services and facilities that use Digital 
Communication Technologies are sufficiently available to people with disability. 


Availability can be reduced where people with disability cannot afford, or do not know, 
about such goods, services and facilities. Socio-economic disadvantage, which is more 
common among people with disability, exacerbates this problem. This form of exclusion 
can also worsen inequality and disadvantage for people with disability.” Barriers to 
availability may be compounded when disability intersects with other factors, including 
being older, having multiple health conditions or one’s geographic location.” 


The problem of availability differs from a problem of functional inaccessibility. For 
example, a smartphone may be designed to enable a person with vision impairment to 
use it, but if they cannot afford the phone they will not benefit from this good design. 


As summarised in Table 3, the Project's consultation revealed three common barriers 
to access for people with disability in this area: understanding and awareness of what 
is available to people with disability (also referred to as digital ability);°*° affordability 
of home internet service; and affordability of critical goods, services and facilities that 
use Digital Communication Technologies. These barriers are often interrelated, and 
exacerbated by socio-economic factors. 


— 
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This chapter recommends measures to support greater availability of goods, services and facilities that use 
Digital Communication Technologies for people with disability through: 


1. the provision of accessible information on how goods, services and facilities can be used by people 
with disability 


2. more accessible broadband internet by introducing a concessional rate for people with disability 


3. improved access to Digital Communication Technologies for people with disability through National 
Disability Insurance Scheme (NDIS) funding. 


Table 3: Digital Communication Technologies: problems of availability 


Internet access 


People with disability experience lower rates of broadband internet access compared with 

the rest of the Australian population. This gap is sometimes called the ‘digital divide’. People 
with disability in regional and remote areas can experience inadequate and unreliable internet 
coverage. 


Cost of assistive technology 


Goods, services and facilities that use assistive technology, including where inaccessible items 
are altered to make them accessible for people with disability, are often costly. For example, 
screen readers, voice activation software and eye-gaze software can be especially expensive. This 
can be problematic for people with disability and older people, as they are more likely to be on 
low incomes, pensions, unemployed or underemployed. 


Digital ability 


The individual characteristics, experiences and preferences of a person with disability can affect 
their ability to engage with and use Digital Communication Technologies effectively. For example, 
people who have not received support and training may have low confidence and skills with tech- 
powered products and services. 


Socio-economic factors 


Barriers described by people with disability are often interrelated with socio-economic factors, 
such as educational attainment, income, and access to health, legal and social support services. 
For example, one in four community legal centre clients has a disability—a cohort who experience 
digital exclusion due to cost of technology and limited ability to access essential services online.?° 
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14.2 Information about goods, 
services and facilities 


RECOMMENDATION 32: Standards 
Australia should develop, in consultation 
with people with disability and other 
stakeholders, an Australian Standard 

or Technical Specification that covers 

the provision of accessible information, 
instructional and training materials to 
accompany consumer goods, services and 
facilities. 


This Australian Standard or Technical 
Specification should inform the 
development of the recommended Digital 
Communication Technology Disability 
Standard under section 31 of the 
Disability Discrimination Act 1992 (Cth) (see 
Recommendation 24). 


People with disability experience barriers in 
accessing information about how goods, services 
and facilities operate and their suitability for people 
with disability. For example, a product may be 
available but a lack of accessible instructions or 
information may mean that a person with disability 
is unaware of the product, or unable to determine 
if it is suitable for their needs. 


The Commission recommends that Standards 
Australia work with industry and the community 
to develop a standard or technical specification 
to ensure the provision of consistent accessible 
information for Digital Communication 
Technologies.”! This could improve the provision 
of two forms of information: 


1. information about the functional accessibility 
of the product. For example, it could state 
whether a product is compatible with a 
screen reader or whether it has raised 
buttons for a person who is blind 


2. instructional and training materials for 
products in an accessible format, such as 
online content (for a person who is blind 
and uses a screen reader), Braille, Easy 
English or large print.%2 


The Commission received positive feedback on this 
proposal from disability representative groups and 
business.’ 


Stakeholders provided examples of everyday goods 
that use Digital Communication Technologies—such 
as personal and household items and assistive 
technologies—which are difficult to use because 

of limited accessible information and instructional 
material.” Accessible information for these goods 
would enable greater autonomy and participation 
for people with disability. 


Stakeholders submitted that they sometimes need 
to purchase a product before finding out whether 
the product will be functionally accessible for their 
needs.” There is generally low awareness and 
understanding of new technologies among people 
with disability,” and their supporters or carers.?°” 
This can make it difficult to keep up to date with 
new technological developments and opportunities 
for their individual circumstances.°8 


For example, the Ability Research Centre stated: 


There is a significant paucity of knowledge about 
what assistive technology is available, and how 

all kinds of technology can be harnessed for the 
specific purpose of assisting people with disability 
... In essence, the sector languishes from a lack 

of clear knowledge about new, often generic 
technology, how it integrates with existing, often 
specialised, technology, how options can be 
meaningfully evaluated, compared and trialled, 
and where to turn for this information.°° 


Standards Australia has indicated that it could 
facilitate the development of an Australian 
Standard or Technical Specification to cover this 
information.°©° Standards Australia noted the 
current gap in this type of guidance, and the 
negative impact this can have on people with 
disability. 
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Standards Australia and disability representative 
groups also highlighted the importance of including 
people with disability and their representatives 

in the development of a new Standard.%*' ACCAN 
and the Digital Gap Initiative submitted that the 
proposed Australian Standard be mandatory for the 
providers of goods to consumers.? 


Conclusion 


When we buy a new product, or access a new 
service or facility, we generally take for granted 

that instructions will be readily available to make 
the good, service or facility useable. Too often this 
assumption does not hold for people with disability. 


This might be because there are no instructions. 
In theory, the absence of any instructions would 
disadvantage all users equally, but people with 
disability are more likely to be disadvantaged 
because certain features, thought to be intuitive, 
might rely on a person being able bodied. Without 
the ability to see a particular product, for example, 
some explanation might be needed about how to 
operate it. 


Where instructions are written on the assumption 
that all users have the same characteristics and 
abilities, this can also be problematic. Even where 

a good, service or facility has been designed for 

use by people of all abilities, the way a person with 
disability uses it might differ from how others use 
it. If the instructions do not explain the alternative 
way that people with disability may operate the 
good, service or facility, those people will be 
disadvantaged and, in a practical sense, might not 
be able to access the thing at all. The same problem 
arises if the instructions are available only ina 
format that is inaccessible for people with disability. 


Digital Communication Technologies often bring 
unavoidable complexity. Hence, it is important 
that goods, services and facilities that use these 
technologies are accompanied by instructions that 
accommodate the needs of people with disability. 


The development of an Australian Standard would 
be a relatively straightforward way of addressing 
this challenge. Information and instructions 
directed towards meeting the needs of people 

with disability would support informed consumer 
choice, and bring added benefits in terms of greater 
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independence and participation through the use of 
these items. 


Some designers and developers are pioneering 
the use of accessible Digital Communication 
Technologies, while others do not appear to 
prioritise accessibility. The role of ‘human rights by 
design’ is considered in the next chapter. There is 
scope, however, for the private sector to improve 
the accessibility of goods, services and facilities at 
the point of sale and use by the consumer. 


Consumer goods, services and facilities should 
come with accessible instructions and dedicated 
information about accessible features. For example, 
a smart TV's accompanying instructions should be 
in an accessible format and include information 
about its accessibility features and functionality. 
Businesses that adopt such measures could benefit 
from increased market share and goodwill among 
the disability community.?° 


The Commission recommends that Standards 
Australia work with industry and the community 
to develop a Standard or Technical Specification 
to ensure the provision of consistent accessible 
information for Digital Communication 
Technologies.°% 


These standards and specifications should support 
people with disability in obtaining goods, services 
and facilities and when finalised, be incorporated 
into the recommended Digital Communication 
Technology Disability Standard under section 31 

of the DDA.°® 


14.3 Internet access 


RECOMMENDATION 33: The NBN 
Co should implement a reasonable 
concessional broadband rate for people 


with disability who are financially 
vulnerable, in consultation with 
them, their representatives and other 
stakeholders. 


The Commission supports the call from many civil 
society and disability representative groups for 

a concessional broadband rate for low-income 
households, to make internet access more 
affordable and improve digital inclusion.°© 


An example of this advocacy is the No Australian Left 
Offline campaign, led by ACCAN. ACCAN estimates 
that a concession to the 2 million Australian 
households on the lowest incomes could be budget 
neutral when offset against outcomes of increased 
take-up of broadband services, including an 
increase in average incomes and creation of new 
businesses.°°7 


There is an opportunity to increase digital inclusion 
for people with disability through the establishment 
of a concessional rate for people with disability 
who are financially vulnerable. This could be an 
extension and expansion of the NBN Education 
Assistance Package or through a new concession 
scheme. 


In assessing any such scheme, consideration should 
be given to the needs of people with disability who 
might be financially vulnerable, such as people who 
are studying, have high support needs or are on 
rehabilitation pathways. 


The internet is a significant gateway for people with 
disability to be connected with services, to give and 
receive information and enjoy their full range of 
human rights, including freedom of expression and 
opinion.%°° 


Internet access as a central element of digital 
inclusion, and the barrier of affordability for people 
with disability, were common themes throughout 
this Project. Digital inclusion rates in 2020 show 
that people with disability experience lower digital 
inclusion, and that affordability has worsened for 
this group over the last six years: 


Australians with disability spend a greater 
proportion of their household income on internet 
access than the Australian average and receive 
less data for each dollar of expenditure than the 
average.” 


The social and economic causes of digital exclusion 
are complex and varied. Rates of inclusion are 
influenced by differences in income, education 
levels, and the geography of socio-economic 
disadvantage.?”' Low internet access is correlated 
strongly with disability, low family income, and 
employment and education status.°” Further, the 
distribution of poverty and inequality in Australia 
means some people start from a position of 
disadvantage when it comes to digital inclusion, 
and there is a real risk that the changing digital 
environment may exacerbate experiences of 
poverty and inequality.” 


The Australian Red Cross submitted: 


Pervasive service digitisation and dependence 
on technology in our private and public lives can 
further disadvantage vulnerable Australians. 
Improving digital inclusion is critical to ensure 
that everyone in our community is empowered 
to participate and contribute. Technology can 
empower people in so many ways - they can 
stay connected and involved with their social 
and community networks, access knowledge and 
services to help them stay well, or link to learning 
and job opportunities.°” 


In supporting the proposal to increase broadband 
access for people with disability, SAS Institute 
Australia said: 


[T]he adoption of digital technology yields 
enormous benefits for society and it is critically 
important to ensure that all members of society 
can participate in those benefits.°”° 


Stakeholders observed that the COVID-19 pandemic 
has highlighted an even greater need for internet 
access, as people with disability rely on digital 
platforms to access carers, disability and health 
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supports during pandemic restrictions.?”° The 
pandemic required an almost immediate transition 
from in-person activities to digital platforms. This 
was reflected in the uptake of higher internet 
speeds, to accommodate increased broadband use 
for work, education and entertainment.?”” 


In recognition of this increased need, NBN Co 
offered a wholesale discount for people who had 
not accessed broadband services prior to March 
2020.78 It recognised that broadband was required 
to enable access to education, and offered the 
Education Assistance Package until January 2021.7? 
Further, some telecommunications businesses 
offered free additional data through mobile or 
broadband plans during some of the COVID-19 
lockdown period.*°° 


People with disability have reported increased 
cost of living throughout the COVID-19 pandemic 
and, in a recent national survey, 91% of people 
with disability and carers who responded said that 
they experienced higher expenses in groceries, 
healthcare, internet and phone, and hygiene/ 
sanitising equipment.”*' 


14.4 Role of the National Disability 
Insurance Scheme 


RECOMMENDATION 34: The 
National Disability Insurance Agency, in 
consultation with people with disability, 
should review its policies regarding 
funding of reasonable and necessary 
supports as those policies apply to 
accessible goods, services and facilities, 
which use Digital Communication 
Technologies and which can be shown 


to enable people with disability to enjoy 
greater independence and participation in 
all areas of life. 


In particular, the NDIA should focus 

on increasing access to internet plans, 
computers, tablets, laptops and 
smartphones and other items that rely on 
Digital Communication Technologies. 


176 


The NDIS plays an important role in ensuring that 
people with disability have access to goods, services 
and facilities that enable participation in the 
community, although most people with disability do 
not participate in the scheme.?8? 


Under the NDIS, people with disability receive 
‘reasonable and necessary’ supports under 
individualised NDIS plans. The National Disability 
Insurance Scheme Act 2013 (Cth) (NDIS Act) provides 
that reasonable and necessary supports for people 
with disability are those that: 


(a) support people with disability to pursue their 
goals and maximise their independence; and 


(b) support people with disability to live 
independently and to be included in the 
community as fully participating citizens; and 


(c) develop and support the capacity of people 
with disability to undertake activities that enable 
them to participate in the community and in 
employment.?% 


NDIS Rules prescribe what will be funded or 
provided and the methods or criteria used by the 
National Disability Insurance Agency (NDIA).°% 
The rules and policies on the funding of assistive 
technology (which include Digital Communication 
Technologies) are complex. For example, the NDIS 
uses four levels to describe the complexity of a 
participant's assistive technology needs.?5 


The NDIS should support improved access to 

the internet and ‘mainstream’ goods, services 

and facilities that use Digital Communication 
Technologies. The NDIS should extend and expand 
its current policies on funding mainstream devices, 
or establish a new policy to support people with 
disability to obtain these items. 


There is some community concern about how the 
NDIS deals with the technology needs of people 
with disability. This is part of a broader debate 
about the extent to which the NDIS should prioritise 
the provision of assistive or accessible technologies. 
Some stakeholders reported that the NDIA takes an 
inconsistent approach when determining whether 
to provide accessible ‘mainstream’ technology, such 
as smartphones, under NDIS plans.°%° 


It was claimed that some NDIS planning regions 
allow the purchase of mainstream technology, such 
as a smartphone; others a lease; and others do not 
allow this at all.?°” Some suggested that individuals 
with greater capacity to articulate their own 

needs, or those who had a planner to help them 
do so, were more likely to receive goods, services 
and facilities that use Digital Communication 
Technologies under an NDIS plan.?%8 These sorts 

of inconsistency were said to cause confusion and 
frustration across the disability sector.%°? 


Stakeholders submitted that those most at risk of 
exclusion from accessible and assistive technology 
are people with disability who are on a low 
income. Blind Citizens Australia stated that the 
use of assistive technology often depends on access 
to internet and ICT devices, which are considered 
‘mainstream’ technology and not funded under the 
NDIS: 


Most people who receive social security as 

their only source of income are living below 

the poverty line. This could well mean that 
assistive technologies that are funded through an 
individual's NDIS plan, are unable to be accessed 
due to unaffordability of ongoing internet 
connection, smart phone infrastructure and data 
plan, computer, or other device.” 


Conclusion 


The NDIS may cover an item of assistive 
technology if it is considered a reasonable and 
necessary support to meet the needs of the 
participant. Until recently, internet access and 
‘mainstream’ goods that use Digital Communication 
Technologies—such as computers, tablets and 
smartphones—were not usually included in NDIS 
plans. 


However, the NDIA modified its approach in 2020 in 
acknowledgement that many face-to-face services 
were suspended during the COVID-19 pandemic, 
and people with disability need mainstream devices 
to support online delivery of their services.°% Until 
28 February 2021, an NDIS participant could use 
their plan to obtain a computer tablet so that they 
can take part in online activities.” 


The Commission welcomes this change in policy, 
but the change is constrained in two important 
ways. First, it is time limited, because it responds 
to the immediate needs associated with the 
COVID-19 pandemic. The Commission urges that 
this time limitation be removed because the 
pandemic merely makes obvious and more acute 
an existing, serious problem. Secondly, devices 
such as smart phones and tablet computers with 
mobile connections cannot be purchased with NDIS 
funding.” This also should be reconsidered. 


Affordability barriers are difficult to overcome for 
people with disability who are on low incomes. They 
may encounter multiple socio-economic barriers 

to access and inclusion. People with disability 
experience higher levels of poverty than other 
Australians, and 38% of Australians living in poverty 
have a disability.°%° 


The NDIS should support improved access to 

the internet and ‘mainstream’ goods, services 

and facilities that use Digital Communication 
Technologies, where they enable other human 
rights for people with disability, and support 
increased independence and participation. The 
NDIS should extend and expand its current policies 
on funding mainstream devices, or establish a new 
policy to support people with disability to obtain 
these items. These amendments would have 
particular benefit for people with disability who are 
financially vulnerable. 
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building 


15.1 Summary 


Good design is central to making goods, services and facilities that use Digital 
Communication Technologies accessible for people with disability. 


A‘human rights by design’ approach involves considering the human rights implications 
of what one is creating at the earliest stages of design and development, and taking 
steps to uphold the rights of end users and others who may be affected. Applying a 
human rights by design approach in the context of Digital Communication Technologies 
not only benefits people with disability who may have particular needs, but it also 
benefits the broader community and the technology sector. 


This chapter recommends three key ways to apply human rights by design in the context 
of Digital Communication Technologies: 


1. the Australian Government should adopt and promote this approach through 
the National Disability Strategy and in the design and delivery of services 

2. human rights by design should be included in education and training—especially 
in the areas of science, technology, engineering and mathematics (STEM) 

3. this approach should be a part of accreditation, ongoing professional 
development, training and capacity building for those involved in designing and 
developing with Digital Communication Technologies. 
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15.2 Design approaches 


RECOMMENDATION 35: The Disability 
Reform Council, through the Disability 
Reform Ministers’ Meeting, should: 


(a) include accessible technology as 
an outcome area in the next National 
Disability Strategy to improve 
access to Digital Communication 
Technologies for people with 
disability 

(b) lead a process for the Australian 
Government and state and 
territory governments to adopt and 
promote human rights by design 

in the development and delivery of 
government services using Digital 
Communication Technologies, and 
monitor progress in achieving this 
aim. 


This section explores some of the benefits 

in promoting human rights by design in the 
development of goods, services and facilities that 
use Digital Communication Technologies. 


The Commission urges the Australian Government 
to include accessible technology in the next 
National Disability Strategy, and promote human 
rights by design in government services that use 
Digital Communication Technologies. 


(a) Designing for people with disability 


Stakeholders across industry and the community 
considered that the whole community benefits 
when technology is accessible for all.°°” For 
example: 


* people with disability benefit through better 
access to these things and, through this, 
they can enjoy greater independence and 
participation 


* other members of the community tend to 
experience simpler functionality 


e the technology industry benefits from 
a bigger consumer market and better 
reputation. 


Stakeholders emphasised that the right of people 
with disability to access technology should be 
foundational, and considered in the earliest 
conceptual, research and design phases. For 
example, Google submitted: 


Access and accessibility need to be considered 
from the beginning of the development cycle in 
order to produce technologies and products that 
contribute positively to people's lives. 


Industry stakeholders provided positive examples 
of accessible design in their production processes, 
and disability advocacy groups also recognised 
the importance that some businesses have placed 
on accessible design.” The Digital Gap Initiative 
emphasised the importance of avoiding the 
mentality of accessibility as an afterthought: 


[l]n instances where accessibility is provided, our 
experience is that accessibility is a design feature 
added on towards the end of the product lifecycle, 
as opposed to being considered in early design 
phases. The consequence is that accessibility is 
not coherently integrated into products, including 
websites, and thus provides an inconsistent 
experience for end users.'°° 


While this Project has focused primarily on human 
rights by design, this is just one of a number of 
similar design-led approaches, which tend to share 
goals such as inclusivity. Other approaches, which 
were raised in the consultation process, include the 
following.'°™ 


1. Universal design aims for products and 
services that are usable by all people, 
including people with disability, to the 
greatest extent possible, without the need 
for adaptation or specialised design.1° 


2. Accessible design aims for independent 
use, specifically by people with disability, 1°% 
and has internationally recognised 
standards considering a range of 
disabilities.'0% 


3. Inclusive design considers the full 
range of human diversity with respect to 
characteristics such as ability, language, 
gender and age, aiming for outcomes 
usable by all people.'°°> 


4. Co-design focuses on the inclusion of 
people with disability in all design phases 
with the goal of producing greater 
accessibility in the final product. This 
can involve people with disability being 
employed or consulted in the design 
process. 


5. Safety by design aims to put user safety 
and rights at the centre of the design, 
development and release of online 
products and services.'°% 


These approaches overlap in their operation and 
conceptual underpinning.’ For example, the 
‘Accessibility Principles for Banking Services’ uses 
principles of universal design, accessible WCAG 2.1 
guidelines, and inclusive design methodology.' 


Stakeholders noted that each approach has its own 
strengths and challenges.'° No single solution 

will meet the accessibility needs of all people with 
disability.1°'? For instance, Adobe submitted: 


Because different users have different needs, it 
is important to acknowledge there is no single 
perfect accessible final result... Design is not 
engineering, and you can't ‘certify’ designers the 
way you can products. The creative phases of 
product design are too abstract to pin down that 
way. What they need to do is create some kind of 
incentive to build it right from the start.'°" 


Stakeholders also noted that designers need to 
appreciate that one person may have several 
accessibility requirements. The Co-Innovation 
Group at the University of Queensland submitted 
that some people 


require both simple, clear design for cognitive 
accessibility, and high-quality audio and features 
to support hearing accessibility. Managing multiple 
streams of accessibility considerations may be 
challenging and may potentially lead to technology 
that is less accessible because of range of choices 
and set up requirements. '°'2 


(b) Human rights by design 


The various design-led approaches described 

above can be seen as all providing elements of 
human rights by design. Most critically, each 
approach focuses on the importance of embedding 
accessibility into the entire technology cycle— 

from concept, research and design, testing and 
production through to implementation and use— 
but focusing especially on the earliest phases of this 
cycle. 


In the Discussion Paper, the Commission proposed 
that human rights by design strategies be adopted 
and promoted by the Australian Government, 
including in service delivery, policy and in the 
National Disability Strategy. There was broad 
support from industry representatives, people 
with disability, and accessibility consultants for this 
proposal.'0'3 


Some stakeholders saw a benefit in human rights 
by design, over other design-led approaches, 
because it can draw attention to rights that might 
otherwise be neglected in the design process, such 
as the rights to privacy and non-discrimination.'°"* 
Three civil society organisations submitted that 
human rights by design principles should form the 
basis of legal obligations.'0"5 


Some key principles that underlie a human rights by 
design approach in this area include: 


* the common goal is functional accessibility 
for people with disability 


e accessibility is considered at all points in the 
product cycle’®"® 


people with disability and their 
representatives are engaged and consulted 
in the design process, including people with 
different disabilities, perspectives, ages and 
backgrounds‘ 


the most effective way to include people 
with disability in design is to have better 
representation of people with disability in 
the technology sector.'°"8 
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(c) Business case for human rights by design 


Stakeholders across many sectors pointed to 

the commercial opportunities for businesses 

that implement human rights by design.'°"? This 
included through the expansion of markets and 
building a stronger reputation for corporate social 
responsibility. 


(i) Expanding markets 


Human rights or inclusive design can be a tool to 
expand commercial opportunities by strengthening 
existing market share, and by creating and 

growing new markets.'°° Research conducted by 
PwC Australia for the Centre for Inclusive Design 
estimates that inclusive design can increase the 
potential market of products and services by three 
to four times.'°2! In a similar vein, the Digital Gap 
Initiative submitted: 


The private sector should be allowed and 
encouraged to innovate and create products 
that are accessible by all or create products that 
bridge the accessibility gap. Accessibility in our 
view is an untapped commercial market which 
should be highlighted to businesses. Almost one 
in five Australians has some form of a disability, 
which represents a substantial target market for 
business. '° 


Delivering innovative products to consumers and 
solving ‘unmet need’ can help businesses tap into 
network effects and increase customer base and 
market share.'°3 The University of Melbourne 
summarised the commercial imperative as being 
that ‘[i]Jn the age of personalisation, a device 
which cannot be used in every environment is not 
competitive’. 104 


In addition, tech-powered products may be initially 
designed for people with disability but end up 
benefiting a broader base of customers. For 
example, captions on television and SMS or text 
messages were both initially developed for people 
who are deaf or have a hearing impairment, but are 
now used by the entire community. 19 


One stakeholder submitted that inclusive design 
is likely to improve the functionality for all users, 
because it encourages designers to be more 
creative and innovative.'°¢ ‘Edge users'—that is, 
people who are not considered to be among the 
mainstream users of a product or service—are 
included in the design process. They are ‘less likely 
to defend and validate a current design that doesn’t 
meet their needs’, generating greater creativity in 
the design process.'°’ This can 
respond not only to disability needs but also 
enhance access for people of all ages and varying 
literacy abilities, thus addressing human rights 
principles regarding non-discrimination and equity 
of access for all.'°7° 


These benefits extend to goods, services and 
facilities that use Al. Stakeholders submitted that 
functionality is enhanced for all users through 
inclusive design as ‘outlier data’ is valued and 
included, informing more adaptive and innovative 
Al systems. '°9 


(ii) Commercial reputation and risk mitigation 


Some stakeholders referred to the value of 

human rights compliant design for shareholders, 
employees and customers through the corporate 
social responsibility lens.1 Businesses with a 
commitment to human rights and social concerns 
can reap benefits across the organisation, including 
better job performance, increased shareholder 
returns, and loyal customer bases.'°" 


Some people with disability noted that businesses 
with a commitment to accessible and inclusive 
practices have good reputations and loyal customer 
bases across the disability community. 132 


Industry and community stakeholders noted that a 
human rights by design strategy can be more cost 
effective than retrofitting accessibility into a product 
after it has been developed and distributed.'° 
Harpur said, where universal design is adopted, 
‘many access barriers are not created in the first 
place and thus the need to engage in retrofitting is 
reduced or eliminated’.1°* 
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(d) Challenges to implementation of human 
rights by design 


Industry and community stakeholders highlighted 
some challenges to the implementation of human 
rights by design strategies in policy and practice. 


The pace of technological development was cited 
as the most pressing concern for industry. In this 
respect, Telstra submitted: 


There are arange of competing interests that 
could make it challenging to implement this in a 
technology business environment, notably the 
rapid evolution of technology and the short time to 
market from initial concepts that are a significant 
competitive advantage in this sector." 


Some observed that the upfront costs of 

human rights by design could be prohibitive 

or a disincentive for business, depending on 

the technology setting and their business 
capabilities.: Commercial considerations can 
outweigh accessible design considerations in 

the fast-paced technology marketplace, where 
businesses are striving to be innovators and market 
leaders.'09” 


It is common for tech businesses to release what is 
known as a minimum viable product. A minimum 
viable product undergoes iterative improvements 
as it is tested and used by consumers, with the 

aim of producing a more effective and refined final 
product.'° If the initial version of the product is not 
accessible for people with disability, they must then 
wait for further accessibility refinements, which are 
not guaranteed.'°°° 


Stakeholders also noted concerns about improving 
STEM professional and educational training on 
human rights by design, as well as increasing 

the employment of people with disability in the 
technology industry. These issues are explored 
below. 


(e) Conclusion 


The Commission has concluded that there are 
significant benefits in promoting human rights 
by design and there is strong support for this 
approach in the design and development of 
goods, services and facilities that use Digital 


Communication Technologies. There are 
opportunities for the Australian Government to 
lead in this space, through policy and practice, 
including service delivery. 


A human rights by design approach would be 

an effective way of fulfilling Australia’s CRPD 
obligations. These include to promote the design, 
development and production of accessible 
information and communication technology, and 
research and development of universally-designed 
goods, services, equipment and facilities. 


The Commission recognises the links among many 
design-led approaches, such as accessible design, 
universal design and inclusive design. In all of 
these approaches, accessibility is both a goal and 

a measure of success in judging the end product.'™" 


Decisions made in the design process are 
unavoidably informed by the individual life 
experiences of those included in that process.' 

In this way, the best and worst of human 
motivations—from kindness and altruism, to 
narrow thinking and prejudice—can be infused into 
the outputs of any design process. 


Introducing human rights by design, and seeking 
out the involvement of people with disability, at the 
earliest stage has a number of benefits. First and 
most importantly, it benefits people with disability 
who rely on accessibility features. In addition, this 
approach also can have benefits for shareholders, 
employees and other customers. A strategy that 
enables accessibility issues to be dealt with early 

is generally more cost effective than retrofitting 
accessible features at a later stage. 


The Commission concludes, taking into account 
Australia’s obligations under the CRPD, and its 
consultation and research, that a human rights by 
design strategy generally emphasises the following 
principles: 


e The primary goal is accessibility to all 
people, to the greatest extent possible, 
without the need for adaptation or 
specialised design.1°? 


People with disability and their 
representatives should be encouraged 
to provide meaningful input in the 
development process, in roles such 

as designers, co-designers and expert 
consultants. 


People with disability and their 
representatives should be encouraged to 
participate in all phases of the development 
process—concept, research, design, 
iterations, testing, production, manufacture 
and upgrades. 


A human rights by design strategy draws on 
principles that underpin the CRPD—individual 
autonomy, independence, non-discrimination, full 
and effective participation and inclusion in society, 
respect for difference, equality of opportunity, 
accessibility, equality of men and women, and 
respect for children.'°4 


The adoption of a human rights by design strategy 
in government policies and procedures would 

be an important step in promoting accessible 
goods, services and facilities that use Digital 
Communication Technologies. 
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The Disability Reform Council, which is now 
convened as the Disability Reform Ministers’ 
Meeting, is a forum for Australia’s federal, state 
and territory governments to progress key national 
reform in disability policy.‘ The Council oversees 
the implementation of the NDIS, and National 
Disability Agreement and National Disability 
Strategy reforms, to support people with disability, 
their families and carers. 


In 2020, the Department of Social Services (DSS) 
released two papers inviting input on the proposed 
outcome areas in the next National Disability 
Strategy.'°46 In its submission to that Department, 
the Commission urged that ‘accessible technology’ 
be included as an additional outcome area, with 
measures to improve accessible and inclusive 
practices across government services.'°4” 


Human rights by design strategies should 

be incorporated into policies at all levels of 
government through the Disability Reform 
Ministers’ Meeting. The DTA has a role in leading 
and shaping whole-of-government policies on 
accessible ICT and digital services and could 
support these efforts. It could also provide 
some guidance for state, territory and local 
governments, '8 


15.3 Education, training and 
Capacity building 


RECOMMENDATION 36: Providers 
of tertiary and vocational education 
should include the principles of human 
rights by design in relevant degree and 
other courses in science, technology, 


engineering and mathematics. The 
Australian Government should engage the 
Australian Council of Learned Academies 
to provide advice on how to achieve this 
aim most effectively within the tertiary 
and vocational sectors. 


RECOMMENDATION 37: Professional 
accreditation bodies for science, 
technology, engineering and mathematics 
should introduce mandatory training 

on human rights by design as part of 
continuing professional development. 


RECOMMENDATION 38: The Australian 
Government should commission 

an expert body to lead the national 
development and delivery of education, 
training, accreditation, and capacity 
building for accessible technology for 
people with disability. 


Goods, services and facilities relying on Digital 
Communication Technologies are too often 
designed and developed in ways that are 
inaccessible for people with disability. There 

are many contributing factors to this problem. 
However, in the Commission's view, greater use 
of a human rights by design strategy would help 
address the problem. 


Human rights by design is still a relatively new 
concept. More could be done to promote this 
strategy through education, training and capacity 
building initiatives. There was strong support 
among stakeholders for human rights by design in 
Digital Communication Technologies production, 
and for building capacity in the technology industry 
to design and develop human rights compliant 
products and services. There was also recognition 
that building the capacity of technology designers 
would provide benefits for the whole community 
through more usable technology. As discussed 
above, there is also a strong business case for 
human rights by design. 


The Commission recommends three areas where 
human rights by design should be promoted in the 
technology sector: 


e tertiary and vocational education and 
training 


* professional development 


* an organisation to lead the national 
development and delivery of education, 


training, accreditation and capacity building. 


The initiatives recommended by the Commission 
would enhance Australia’s compliance with 
Article 9(2)(c) of the CRPD to provide training on 
accessibility issues, and they would also help 
businesses meet their obligations under the 

UN Guiding Principles on Business and Human 
Rights.1°? 


(a) Education and training 


Stakeholders from industry, academia and civil 
society gave strong support for a proposal to 
introduce human rights by design principles 

into relevant STEM tertiary and vocational 
courses.'°°° Many of these stakeholders suggested 
this coursework should be considered a core 
component for students. 


There was also strong support for the proposal 
that professional accreditation bodies for science, 
technology and engineering introduce mandatory 
training on human rights by design.'°" 


Core components of formal education and 
professional development present the best 

training and intervention opportunities to target 
the workforce responsible for technological design 
and development.'° These education and training 
opportunities would help the workforce understand 
accessibility issues and barriers for people with 
disability, and how they might play a role in 
improving accessible and inclusive design.'°? 


It was suggested that educational and professional 
development coursework on human rights by 
design be developed and delivered with the 
involvement of people with disability. It could 
include content on: understanding disabilities, 
international human rights obligations (including 
the CRPD and the UN Guiding Principles on 
Business and Human Rights), design methods, 


internationally recognised standards and best 
practice.'°> 


To this end, the Australian Council of Learned 
Academies (ACOLA) observed: 


Incorporating ‘human rights by design’ principles, 
alongside initiatives that support a diverse 
workforce to design emerging technologies, will 
facilitate opportunities for access and uptake 
of emerging technologies by all members of 
our community and provide scope to overcome 
existing societal inequalities. If programmed 
with these principles from the beginning, we 
can facilitate beneficial and safe integration and 
opportunities between humans and emerging 
technologies. '0%° 


Several stakeholders urged that a human rights 
by design course be included in the curricula of 
other tertiary courses, such as communications, 
media, business, law, human resources and social 
sciences.'%” They noted the intersection of these 
disciplines with the use of new and emerging 
technologies, such as the use of Al in hiring 
practices. However, many of these stakeholders 
noted the importance of prioritising science, 
technology and engineering students when 
considering the design and delivery of accessible 
goods, services and facilities to people with 
disability.10°° 


(b) Awareness and capacity building 


There was strong stakeholder support for the 
proposal that an organisation be commissioned 
by the Australian Government to lead the national 
development and delivery of education, training, 
accreditation and capacity building for accessible 
technology for people with disability.1°°° 


Stakeholders recognised a gap in these important 
roles of increasing awareness across public and 
private providers of goods, services and facilities 
that use Digital Communication Technologies. 
The Co-Innovation Group at the University of 
Queensland stated: 


While many express willingness to create 
accessible and ethical technologies, they lack 
knowledge about both ethical frameworks and 
considerations and accessibility related needs 
and approaches. '°° 
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Speech Pathology Australia highlighted the 
importance of raising understanding and 
awareness in areas where there are known to be 
significant Communication barriers for people with 
disability—especially the justice system, primary 
health services and general practice, hospital 
systems, aged care systems, and local government 
consumer-facing services.'°' Vision Australia noted 
significant barriers for people who are blind and 
studying at university and submitted that there is 
a need for capacity building across the education 
sector. 1° 


A key concern was a lack of organisational 
knowledge and skills.1°° In this respect, one 
industry stakeholder said: 


There is an appetite to build accessible platforms 
and technology—the biggest issue we have is 
getting clarity on all the various things we need to 
do. There are so many places to go to, but industry 
needs one guiding source of information.'° 


The Public Interest Advocacy Centre noted the need 
for a consistent approach to accessibility standards 
across industry and the private sector, to avoid 
products like the Commonwealth Bank ‘Albert’ 
EFTPOS touch screen machine, which was not able 
to be used by some people who are blind or have a 
vision impairment. 12 


Some stakeholders referred to the difficulties of 
high turnover across large organisations, and the 
complexity of issues where several teams are 
involved in procurement and operational decision- 
making: 


We need a better culture of ‘human rights by 
design’ throughout business, but it’s a huge 
challenge. There is a high turnover of staff, and 

| spend so much time re-training people across 
the organisation—designers, analysts, testers— 
plus the constant problem of procuring accessible 
products. 1% 


Microsoft submitted that a broader cultural change 
across industry is needed. In this context, a leading 
organisation could be involved in encouraging 


senior executive awareness and board level 
support for accessibility and growing the support 
available to organisations seeking to implement 
good practice—such as accessibility design and 
development expertise—which is in relatively 
short supply in Australia.'°° 


Stakeholders gave strong support to the proposal 
that an organisation be commissioned by the 
Australian Government to lead the national 
development and delivery of education, training, 
accreditation and capacity building for accessible 
technology for people with disability.1°° Industry 
representatives and accessibility consultants 
recognised a significant need for such an 
organisation, to help build capacity and support 
accessible practices across industry. 


(c) People with disability in the workplace 


People with disability will have more opportunities 
for meaningful input into human rights by design 
work if they have greater representation in the 
technology sector. The benefits of this were 
highlighted by stakeholders. 


First, people with disability can directly inform the 
design of new and emerging technologies in their 
work.'°? SAS Institute Australia noted this is the 
most effective practice for embedding human rights 
into design practices: 


it is useful to include people with disabilities in the 
design process. It is much better for that process 
to be led by people with disabilities.107° 


Secondly, stakeholders submitted that a positive 
workplace culture change towards people with 
disability and accessible design is best achieved 
through the employment of people with disability 
at work.'°”' Designers are more likely to consider 
the accessibility needs of people with disability if 
people with lived experience of disability form part 
of design teams and the broader businesses or 
organisations they sit within.'°72 


SAS Institute Australia stated that their efforts 

to create accessible products were only possible 
when they had invested in building the capacity of 
their workforce through cross-team training and 
collaboration. People with disability are essential 
contributors to this activity.'°”? 
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Thirdly, stakeholders noted the reputational 

and corporate social responsibility benefits for 
businesses employing people with disability. These 
include positive links between organisational 
commitment to the social cause, high levels of in- 
role job performance,'°” and increased shareholder 
returns from a more diverse employee and board 
membership. '°75 


(d) Conclusion 


The Commission focuses on three areas where 
human rights by design should be promoted in the 
technology sector: 


e tertiary and vocational education and 
training 


* professional development 


* an organisation to lead the national 
development and delivery of education, 
training, accreditation and capacity building. 


(i) Tertiary and vocational education 
and training 


Tertiary and vocational students of science, 
technology and engineering are or will become the 
primary designers of goods, services and facilities 
that use Digital Communication Technologies. 
Others are also involved in the implementation 

and deployment of these technologies. However, 
targeting scientists, technologists and engineers as 
a priority for courses relating to human rights by 
design will help raise the importance of these issues 
more broadly across educational contexts. 


A human rights by design course could cover 
different models of disability (for example, social, 
legal and medical), international and national legal 
frameworks such as the CRPD and DDA, accessible 
design methodologies, and best industry practice. 


ACOLA brings together four independent learned 
academies: the humanities; science; social sciences; 
and technology and engineering.'°”° ACOLA helps 
inform national policy through various activities, 
including coordinating multi-stakeholder groups 
and consulting on significant national issues. 


ACOLA has indicated that it would welcome the 
opportunity to facilitate human rights by design 
coursework with appropriate support.'°”” Such 
courses should be developed in consultation with 
people with disability and their representatives, 
professional engineers, technologists and 
scientists, the Department of Education, Skills and 
Employment, and the tertiary education sector. 


(ii) Professional development 


Education and training on human rights by design 
would benefit professionals who are already 
practising as designers and engineers. Such 
training could be offered to Chartered Professional 
Engineers and Technologists, for example, who 

are required to undertake 150 hours of continuing 
professional development (CPD) over three years to 
maintain their Chartered Status.'°”8 


CPD activities support an engineer as they carry 
out their technical and professional duties, through 
conferences and training courses, and also allows 
for the completion of tertiary or post-graduate 
courses.107° 


The current CPD requirements include a minimum 
number of hours to be dedicated to the engineer's 
area of practice, risk management and business 
and management skills. A human rights by design 
course should be a minimum requirement within 
an engineer's three-year CPD cycle. 


(iii) An education, training, accreditation and 
capacity building organisation 


There should be targeted capacity building across 
the technology sector and a role for accessibility 
accreditation. 


There was broad agreement across industry, 
government and the community about a lack of 
understanding in the technology industry of the 
access rights of people with disability. This problem 
could be addressed by encouraging developers of 
Digital Communication Technologies to incorporate 
human rights by design principles through existing 
frameworks, such as corporate social responsibility. 


There were two main activities supported by 
stakeholders as vital for industry: 


e education, training and capacity building for 
public and private entities on human rights 
by design 


e the creation of an accessibility accreditation 
scheme to support organisations 
implement and achieve nationally 
standardised accessibility benchmarks. 


Capacity building in the technology sector would 
improve awareness and understanding of the right 
of people with disability to access technology, and 
the obligation to design and develop human rights 
compliant products and services. 


An expert body tasked with capacity building and 
accreditation roles could also support education 
and training efforts such as the development 
of a human rights by design professional and 
educational unit of study. 


The Commission does not have a strong view on 
what entity should take up this role. The Digital 
Transformation Agency and the Department of 
Industry, Science, Energy and Resources have 
experience in supporting accessible policies and 
practices, and capacity building and support in 
industry.'°°° They could work with the Disability 
Reform Council, at Disability Reform Ministers’ 
Meetings, to determine a suitable organisation 
for the role. 


(iv) Employment of people with disability 


There are clear benefits in increasing the 
employment of people with disability in the design 
and development of new technologies. 


Goods, services and facilities that use Digital 
Communication Technologies are likely to be more 
accessible when designers have lived experience of 
disability. In addition, businesses may realise other 
benefits from employing of people with disability, 
such as enhanced reputation and brand, and 
improved employee and customer retention.'" 


The employment rate for working-age people with 
disability is 48%, which is lower than the those 
without disability (80%); and people with disability 
are more likely to be employed part-time than the 
rest of the population.'° 


The Commission is examining measures to increase 
employment opportunities for people with disability 
in its ‘IncludeAbility’ project, led by the Disability 
Discrimination Commissioner, Dr Ben Gauntlett. 
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PART A: NATIONAL STRATEGY ON NEW AND EMERGING TECHNOLOGIES 


Recommendation 1: The Digital Australia Strategy, which is currently being developed 
by the Australian Government Department of the Prime Minister and Cabinet, should 
set Australia’s national strategy for new and emerging technologies. The Digital Australia 
Strategy should promote responsible innovation through: 


(a) effective regulation—including law, co-regulation and self-regulation—that 
upholds human rights in the development and use of new technologies 


(b) the development of a community-wide action plan on education, training and 
capacity building regarding the human rights implications of new and emerging 
technologies 


(c) funding and investment for responsible innovation that complies with human 
rights 


(d) practical measures to achieve the Strategy’s aims, including through the 
establishment of an Al Safety Commissioner (see Recommendation 22). 


PART B: ARTIFICIAL INTELLIGENCE 
Chapter 5 Legal accountability for government use of Al 


Recommendation 2: The Australian Government should introduce legislation to require 
that a human rights impact assessment (HRIA) be undertaken before any department or 
agency uses an Al-informed decision-making system to make administrative decisions. 


An HRIA should include public consultation, focusing on those most likely to be affected. 
An HRIA should assess whether the proposed Al-informed decision-making system: 


(a) complies with Australia’s international human rights law obligations 


(b) will involve automating any discretionary element of administrative decisions, 
including by reference to the Commonwealth Ombudsman's Automated decision- 
moking better practice guide and other expert guidance 


(c) provides for appropriate review of decisions by human decision makers 


(d) is authorised and governed by legislation. 


Recommendation 3: The Australian Government should introduce legislation to 
require that any affected individual is notified where artificial intelligence is materially 
used in making an administrative decision. That notification should include information 
regarding how an affected individual can challenge the decision. 


Recommendation 4: The Australian Government should commission an audit of all 
current or proposed use of Al-informed decision making by or on behalf of Government 
agencies. The Al Safety Commissioner (see Recommendation 22), or another suitable 
expert body, should conduct this audit. 
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Recommendation 5: The Australian Government 
should not make administrative decisions, 
including through the use of automation or artificial 
intelligence, if the decision maker cannot generate 
reasons or a technical explanation for an affected 
person. 


Recommendation 6: The Australian Government 
should make clear that, where a person has a 

legal entitlement to reasons for a decision, this 
entitlement exists regardless of how the decision is 
made. To this end, relevant legislation including s 
25D of the Acts Interpretation Act 1901 (Cth) should 
be amended to provide that: 


(a) for the avoidance of doubt, the term 
‘decision’ includes decisions made using 
automation and other forms of artificial 
intelligence 


(b) where a person has a right to reasons 
the person is entitled also to a technical 
explanation of the decision, in a form that 
could be assessed and validated by a person 
with relevant technical expertise 


(c) the decision maker must provide this 
technical explanation to the person within a 
reasonable time following any valid request. 


Recommendation 7: The Australian Government 
should engage a suitable expert body, such as the 
Al Safety Commissioner (see Recommendation 
22), to develop guidance for government and non- 
government bodies on how to generate reasons, 
including a technical explanation, for Al-informed 
decisions. 


Recommendation 8: The Australian Government 
should introduce legislation to create or ensure 

a right to merits review, generally before an 
independent tribunal such as the Administrative 
Appeals Tribunal, for any Al-informed 
administrative decision. 


Chapter 6 Legal accountability for private 
sector use of Al 


Recommendation 9: The Australian Government's 
Al Ethics Principles should be used to encourage 
corporations and other non-government bodies 

to undertake a human rights impact assessment 
before using an Al-informed decision-making 
system. The Government should engage the Al 
Safety Commissioner (Recommendation 22) to 
issue guidance for the private sector on how to 
undertake human rights impact assessments. 


Recommendation 10: The Australian Government 
should introduce legislation to require that any 
affected individual is notified when a corporation or 
other legal person materially uses Al in a decision- 
making process that affects the legal, or similarly 
significant, rights of the individual. 


Recommendation 11: The Australian Government 
should introduce legislation that provides a 
rebuttable presumption that, where a corporation 
or other legal person is responsible for making a 
decision, that legal person is legally liable for the 
decision regardless of how it is made, including 
where the decision is automated or is made using 
artificial intelligence. 


Recommendation 12: Centres of expertise, 
including the newly established Australian Research 
Council Centre of Excellence for Automated 
Decision-Making and Society, should prioritise 
research on the ‘explainability’ of Al-informed 
decision making. 


Recommendation 13: The Australian Government 
should introduce legislation to provide that where a 
court, or regulatory, oversight or dispute resolution 
body, has power to order the production of 
information or other material from a corporation or 
other legal person: 


(a) for the avoidance of doubt, the person 
must comply with this order even where the 
person uses a form of technology, such as 
artificial intelligence, that makes it difficult to 
comply with the order 


(b) if the person fails to comply with the order 
because of the technology the person uses, 
the body may draw an adverse inference 
about the decision-making process or other 
related matters. 


Chapter 7 Encouraging better Al-informed 
decision making 


Recommendation 14: The Australian Government 
should convene a multi-disciplinary taskforce on Al- 
informed decision making, led by an independent 
body, such as the Al Safety Commissioner 
(Recommendation 22). The taskforce should: 


(a) promote the use of human rights by 
design in this area 


(b) advise on the development and use 
of voluntary standards and certification 
schemes 


(c) advise on the development of one or more 
regulatory sandboxes focused on upholding 
human rights in the use of Al-informed 
decision making. 


The taskforce should consult widely in the public 
and private sectors, including with those whose 
human rights are likely to be significantly affected 
by Al-informed decision making. 


Recommendation 15: The Australian Government 
should appoint an independent body, such as the 
Al Safety Commissioner (Recommendation 22), 

to develop a tool to assist private sector bodies 
undertake human rights impact assessments 
(HRIAs) in developing Al-informed decision-making 
systems. The Australian Government should 
maintain a public register of completed HRIAs. 


Recommendation 16: The Australian Government 
should adopt a human rights approach to 
procurement of products and services that use 
artificial intelligence. The Department of Finance, in 
consultation with the Digital Transformation Agency 
and other key decision makers and stakeholders, 
should amend current procurement law, policy 
and guidance to require that human rights are 
protected in the design and development of any 
Al-informed decision-making tool procured by the 
Australian Government. 


Recommendation 17: The Australian Government 
should engage an expert body, such as the Al Safety 
Commissioner (Recommendation 22), to issue 
guidance to the private sector on good practice 
regarding human review, oversight and monitoring 
of Al-informed decision-making systems. This body 
should also advise the Government on ways to 
incentivise such good practice through the use of 
voluntary standards, certification schemes and 
government procurement rules. 


Chapter 8 Al, equality and non- 
discrimination 


Recommendation 18: The Australian Government 
should resource the Australian Human Rights 
Commission to produce guidelines for government 
and non-government bodies on complying with 
federal anti-discrimination laws in the use of Al- 
informed decision making. 
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Chapter 9 Biometric surveillance, facial 
recognition and privacy 


Recommendation 19: Australia’s federal, state and 
territory governments should introduce legislation 
that regulates the use of facial recognition and 
other biometric technology. The legislation should: 


(a) expressly protect human rights 


(b) apply to the use of this technology in 
decision making that has a legal, or similarly 
significant, effect for individuals, or where 
there is a high risk to human rights, such as in 
policing and law enforcement 


(c) be developed through in-depth 
consultation with the community, industry 
and expert bodies such as the Australian 
Human Rights Commission and the Office of 
the Australian Information Commissioner. 


Recommendation 20: Until the legislation 
recommended in Recommendation 19 comes 

into effect, Australia’s federal, state and territory 
governments should introduce a moratorium on 
the use of facial recognition and other biometric 
technology in decision making that has a legal, or 
similarly significant, effect for individuals, or where 
there is a high risk to human rights, such as in 
policing and law enforcement. 


Recommendation 21: The Australian Government 
should introduce a statutory cause of action for 
serious invasion of privacy. 


PART C: SUPPORTING EFFECTIVE 
REGULATION 


Recommendation 22: The Australian Government 
should establish an Al Safety Commissioner as 

an independent statutory office, focused on 
promoting safety and protecting human rights in 
the development and use of Al in Australia. The Al 
Safety Commissioner should: 


(a) work with regulators to build their 
technical capacity regarding the development 
and use of Al in areas for which those 
regulators have responsibility 


(b) monitor and investigate developments 
and trends in the use of Al, especially in areas 
of particular human rights risk 


(c) provide independent expertise relating 
to Al and human rights for Australian policy 
makers 


(d) issue guidance to government and the 
private sector on how to comply with laws 
and ethical requirements in the use of Al. 


Recommendation 23: The Al Safety Commissioner 
(see Recommendation 22) should: 


(a) be independent from government in its 
structure, operations and legislative mandate, 
but may be incorporated into an existing 
body or be formed as a new, separate body 


(b) be adequately resourced, wholly or 
primarily by the Australian Government 


(c) be required to have regard to the 
impact of the development and use of Al 
on vulnerable and marginalised people in 
Australia 


(d) draw on diverse expertise and 
perspectives including by convening an Al 
advisory council. 


PART D: ACCESSIBLE TECHNOLOGY 
Chapter 12 Functional accessibility 


Recommendation 24: The Attorney-General 
should: 


(a) develop a Digital Communication 
Technology Standard under section 31 of the 
Disability Discrimination Act 1992 (Cth), and 


(b) consider other law and policy reform 

to implement the full range of accessibility 
obligations regarding Digital Communication 
Technologies under the Convention on the 
Rights of Persons with Disabilities. 


In doing so, the Attorney-General should consult 
widely, especially with people with disability and the 
technology sector. 


Recommendation 25: The Australian Government 
and state, territory and local governments should 
commit to using Digital Communication Technology 
that fully complies with recognised accessibility 
standards—especially WCAG 2.1 and Australian 
Standard EN 301 549, and successor standards. To 
this end, all Australian governments should: 


(a) introduce whole-of-government 
requirements for compliance with these 
standards, including by: 


e providing information that is publicly 
available about how each agency 
complies with these requirements, 
reported annually 


establishing central line agency and 
ministerial responsibility for monitoring 
compliance across government 


resourcing training and advisory support 
to assist compliance 


(b) promote accessible goods, services and 
facilities that use Digital Communication 
Technology by favouring procurement from 
entities that implement such accessibility 
standards in their own activities 


(c) develop policies and targets to increase the 
availability of government communications 

in Easy English and provide human customer 
supports for people with disability who need 
to communicate with people instead of 
accessing digital services. 


Recommendation 26: The Australian Government 
Department of Industry, Science, Energy and 
Resources or the Digital Transformation Agency 
should conduct an inquiry into compliance by 
industry with accessibility standards such as WCAG 
2.1 and Australian Standard EN 301 549. 


The inquiry should consider the extent to which 
incentives for compliance with standards should 
include changes relating to taxation, grants and 
procurement, research and design, and the 
promotion of good practices by industry. 


Chapter 13 Broadcasting and audio-visual 
services 


Recommendation 27: The Australian Government 
should amend the Broadcasting Services Act 1992 
(Cth) to increase the amount of accessible content 
available for people who have hearing or vision 
difficulties as follows: 


(a) national and commercial free-to-air 
television services should be required 

to provide audio described content for a 
minimum of 14 hours of programming per 
week, distributed across the primary and 
secondary channels. This should be increased 
to a minimum of 21 hours per week in a 
timeframe to be determined in consultation 
with people with disability and broadcasting 
services. 


(b) subscription television services should be 
required to provide audio described content 
for a minimum of 14 hours of programming 
per week for their main channels. This should 
be increased to a minimum of 21 hours per 
week in a timeframe to be determined in 
consultation with people with disability and 
broadcasting services. 


(c) national and commercial television 
free-to-air services should be required to 
increase the captioning of their content 

on an annual basis, resulting in all such 
broadcasting being captioned on primary and 
secondary channels within five years. The 
Government should determine a formula for 
annual progressive increases of captioning 

in consultation with industry, people with 
disability and their representatives. 


Recommendation 28: The Australian Government 
Department of Infrastructure, Transport, Regional 
Development and Communications should 
conduct a review to identify effective, practical 
ways to increase audio description and captioning 
on secondary or specialist broadcast television 
channels. 
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Recommendation 29: The Australian Government 
should introduce legislation to provide minimum 
requirements for audio description and captioning 
in respect of audio-visual content delivered through 
subscription video-on-demand, social media 

and other services that are not covered by the 
Broadcasting Services Act 1992 (Cth). Obligations 
should be determined in consultation with industry, 
and people with disability and their representatives. 


Recommendation 30: The Australian Government, 
and state and territory governments, should 

ensure that people with disability can receive and 
understand emergency and other important public 
announcements, including by requiring government 
agencies to provide Auslan interpreters at their 
emergency and important public announcements. 


The Australian Government should amend the 
Broadcasting Services Act 1992 (Cth) to require any 
television or other company, which broadcasts 

or re-broadcasts emergency and other important 
public announcements, to ensure that Auslan 
interpretation is visible on the screen at all relevant 
times; and captions are readable, accurate and 
comprehensible. 


Recommendation 31: The Australian 
Communications and Media Authority should 
consult with broadcasters and introduce monitoring 
and compliance measures to support them to: 


(a) comply with accessible service 
requirements 


(b) provide quality accessible services 


(c) increase organisational capacity to comply 
with current and future accessible service 
obligations. 


Chapter 14 Availability of new technology 


Recommendation 32: Standards Australia should 
develop, in consultation with people with disability 
and other stakeholders, an Australian Standard or 
Technical Specification that covers the provision of 
accessible information, instructional and training 
materials to accompany consumer goods, services 
and facilities. 


This Australian Standard or Technical Specification 
should inform the development of the 
recommended Digital Communication Technology 
Disability Standard under section 31 of the Disability 
Discrimination Act 1992 (Cth) (see Recommendation 
24). 


Recommendation 33: The NBN Co should 
implement a reasonable concessional broadband 
rate for people with disability who are financially 
vulnerable, in consultation with them, their 
representatives and other stakeholders. 


Recommendation 34: The National Disability 
Insurance Agency, in consultation with people 
with disability, should review its policies regarding 
funding of reasonable and necessary supports as 
those policies apply to accessible goods, services 
and facilities, which use Digital Communication 
Technologies and which can be shown to 

enable people with disability to enjoy greater 
independence and participation in all areas of life. 


In particular, the NDIA should focus on increasing 
access to internet plans, computers, tablets, laptops 
and smartphones and other items that rely on 
Digital Communication Technologies. 


Chapter 15 Design, education and capacity 
building 


Recommendation 35: The Disability Reform 
Council, through the Disability Reform Ministers’ 
Meeting, should: 


(a) include accessible technology as an 
outcome area in the next National Disability 
Strategy to improve access to Digital 
Communication Technologies for people with 
disability 

(b) lead a process for the Australian 
Government and state and territory 
governments to adopt and promote human 
rights by design in the development and 
delivery of government services using Digital 
Communication Technologies, and monitor 
progress in achieving this aim. 


Recommendation 36: Providers of tertiary and 
vocational education should include the principles 
of human rights by design in relevant degree and 
other courses in science, technology, engineering 
and mathematics. The Australian Government 
should engage the Australian Council of Learned 
Academies to provide advice on how to achieve 
this aim most effectively within the tertiary and 
vocational sectors. 


Recommendation 37: Professional accreditation 
bodies for science, technology, engineering and 
mathematics should introduce mandatory training 
on human rights by design as part of continuing 
professional development. 


Recommendation 38: The Australian Government 
should commission an expert body to lead the 
national development and delivery of education, 
training, accreditation, and capacity building for 
accessible technology for people with disability. 
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Appendix C: Acronyms used in this 
Final Report 


AAT Act Administrative Appeals Tribunal Act 1975 (Cth) 


ACCAN Australian Communications Consumer Action Network 

ACCC Australian Competition and Consumer Commission 

ACMA Australian Communications and Media Authority 

ACOLA Australian Council of Learned Academies 

ADJR Act Administrative Decisions (Judicial Review) Act 1977 (Cth) 

ALRC Australian Law Reform Commission 

Al artificial intelligence 

AR augmented reality 

ASIC Australian Securities and Investment Commission 

ATO Australian Taxation Office 

CAT Convention against Torture and Other Cruel, Inhuman or Degrading 
Treatment or Punishment 

CEDAW Convention on the Elimination of All Forms of Discrimination against 
Women 

CPD continuing professional development 

CRC Convention on the Rights of the Child 

CRPD Convention on the Rights of Persons with Disabilities 

DDA Disability Discrimination Act 1992 (Cth) 

DISER Department of Industry, Science, Energy and Resources 

DP Human Rights and Technology Discussion Paper 

DTA Digital Transformation Agency 

EU European Union 

GDPR General Data Protection Regulation (European Union) 

HRIA human rights impact assessment 

ICCPR International Covenant on Civil and Political Rights 

ICERD International Convention on the Elimination of All Forms of Racial 
Discrimination 

ICESCR International Covenant on Economic, Social and Cultural Rights 

ICT information and communications technology 
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loT Internet of Things 

IP Human Rights and Technology Issues Paper 

NDIA National Disability Insurance Agency 

NDIS National Disability Insurance Scheme 

OAIC Office of the Australian Information Commissioner 
OECD Organisation for Economic Cooperation and Development 
STEM science, technology, engineering and maths 

SVOD subscription video-on-demand 

UDHR Universal Declaration of Human Rights 

UK ICO United Kingdom Information Commissioner's Office 
UN United Nations 

VR virtual reality 

WCAG Web Content Accessibility Guidelines 

WP Human Rights and Technology White Paper 
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der Nederlanden (SyRI), District Court of the Hague, Case No 
C/09/550982/HA ZA 18/388 (5 February 2020) [8]-[9]. 

Brief by the UN Special Rapporteur on Extreme Poverty and 
Human rights as Amicus Curiae in the Case of NJCM c.s./De Staat 
der Nederlanden (SyRI), District Court of the Hague, Case No 
C/09/550982/HA ZA 18/388 (5 February 2020) [24]-[25]. 

Brief by the UN Special Rapporteur on Extreme Poverty and 
Human Rights as Amicus Curiae in the Case of NJCM c.s./De Staat 
der Nederlanden (SyRI), District Court of the Hague, Case No 
C/09/550982/HA ZA 18/388 (5 February 2020) [22]. See also Philip 
Alston, Report of the Special Rapporteur on Extreme Poverty and 
Human Rights, UN Doc A/74/493 (11 October 2019) [6]. 

Australian Government, Department of Industry, Innovation and 
Science, Al Ethics Framework (November 2019). 

Council of Europe Commissioner for Human Rights, Unboxing 
Artificial Intelligence: 10 Steps to Protect Human Rights (May 2019) 7. 
NSW Government, Building Public Trust (4 September 2020) <https:// 
www.digital.nsw.gov.au/policy/artificial-intelligence-ai/ai-strategy/ 
building-public-trust>. 

Privacy Act 1988 (Cth) s 33D. 

Under legislation passed in 2018, modern slavery statements— 
describing the risks of modern slavery in the operations and supply 
chains, and actions taken to address those risks—must be provided 
to the Minister by specific entities: Modern Slavery Act 2078 (Cth) Part 
2. 

See, for example, Human Rights (Parliamentary Scrutiny) Act 2011 
(Cth) ss 8, 9(1); Legislation Act 2003 (Cth) s 15)(2)(f). 

Privacy Act 1988 (Cth) s 33D. 

Under legislation passed in 2018, modern slavery statements— 
describing the risks of modern slavery in the operations and supply 
chains, and actions taken to address those risks—must be provided 
to the Minister by specific entities: Modern Slavery Act 2078 (Cth) pt 2. 


281 


282 


283 


284 


285 


286 


287 


288 


289 
290 


291 


292 


293 
294 
295 


296 


297 


298 


299 


300 


301 


The Attorney-General’s Department has provided guidance on 

this process. See Australian Government, Attorney-General’s 
Department, Statements of Compatibility <https://www.ag.gov.au/ 
rights-and-protections/human-rights-and-anti-discrimination/ 
human-rights-scrutiny/statements-compatibility>. 

Monika Zalnieriute, Lyria Bennett Moses and George Williams, ‘The 
Rule of Law and Automation of Government Decision-making’ (2019) 
82(3) Modern Law Review 425, 429. 

Toby Walsh, Neil Levy, Genevieve Bell, Anthony Elliott, James 
Maclaurin, Iven Mareels and Fiona Wood, The Effective and Ethical 
Development of Artificial Intelligence: An Opportunity to Improve our 
Wellbeing (Report for the Australian Council of Learned Academies, 
July 2019) 10. 

Australian Government, Department of Industry, Innovation and 
Science, Al Ethics Framework (November 2019). 

Australian Human Rights Commission, Human Rights and Technology 
(Discussion Paper, December 2019) Proposal 5. 

DP submissions: Al Now Institute, 4; Australian Academy of 

Science, 3; Australian Privacy Foundation, Queensland Council 

for Civil Liberties, Liberty Victoria, Electronic Frontiers Australia, 
NSW Council for Civil Liberties, 6; Community and Public Sector 
Union, 1; Consumer Policy Research Centre, 13; Digital Rights 
Watch, 3; Element Al, 3; Ethics Matters Pty Ltd, 3; Information 

and Privacy Commission NSW, 10; J Powles, W Bateman, 3; Law 
Council of Australia, 14; Maurice Blackburn Lawyers, 12; Microsoft, 
5; Medical Insurance Group Australia, 3; NSW Bar Association, 7; 
NSW Council for Civil Liberties, 5; Office of the Victorian Information 
Commissioner, 4; WiseLaw, 3; University of Technology Sydney, 17. 
DP submissions: Element Al, 1; Maurice Blackburn Lawyers; Access 
Now, 4; Victoria Legal Aid, 1-2; SAS Institute Australia, 7. 

DP submissions: Al Now Institute, 4; Australian Industry Group, 6; J 
Powles, W Bateman, 3; K Weatherall, T Caetano, 6; KPMG Australia, 
21; R Chalmers, 4; SAS Institute Australia, 7; W Small, 1; University 
of Technology Sydney, 17; Digital Industry Group, 13; Digital Rights 
Watch, 3. 

DP submission: Digital Industry Group Inc, 9. 
DP submissions: Australian Academy of Science, 3; Law Council of 
Australia, 14 (stated the cost of this proposal is potentially onerous, 
but noted ‘That does not detract from its value’); WiseLaw, 4 (noting 
the potential cost but endorsing the proposal). 

DP submissions: Commonwealth Bank of Australia, 4; Microsoft, 

5; Office of the Australian Information Commissioner, 7; Medical 
Insurance Group Australia, 3. 

DP submissions: Aboriginal and Torres Strait Islander Legal Services 
(QLD), 2; Australian Industry Group, 3; Liberty Victoria, Australian 
Privacy Foundation, Queensland Council for Civil Liberties, Electronic 
Frontiers Australia, NSW Council for Civil Liberties, 13; Consumer 
Policy Research Centre, 10; Ethics Matters Pty Ltd, 3; K Weatherall, 
T Caetano, 7; KPMG Australia, 26; Maurice Blackburn Lawyers, 18; 
Office of the Victorian Information Commissioner, 5; The Australia 
Institute’s Centre for Responsible Technology, 10; WiseLaw, 9. 

DP submission: Consumer Policy Research Centre, 10. 

DP submission: Microsoft, 6. 

DP submissions: Maurice Blackburn Lawyers, 6; K Weatherall, 

T Caetano, 6. 

59% of respondents said it was ‘very important’ to be informed 

that a computer program has been used to make an automated 
decision, and 26% said it was ‘somewhat important’: Essential 
Media, The Essential Report—Human Rights Commission (29 July 2020). 
The Australian Government's ‘Digital Business Plan’ outlines 
Government investment of close to $800 million in digital 
technology as part of its economic recovery plan: Prime Minister 

of Australia, ‘Digital Business Plan to Drive Australia’s Economic 
Recovery’ (Media Release, 29 September 2020). 

Australian Government, Department of Industry, Science, Energy 
and Resources, An Al Action Plan for all Australians: A Call for Views - 
Discussion Paper (2020) 4. 

New Zealand Government, Stats NZ, Algorithm Assessment Report 
(October 2018). 

UK Government, Centre for Data Ethics and Innovation, A/ Barometer 
Report (June 2020). 

Al Now Institute, Confronting Black Boxes: A Shadow Report of the New 
York City Automated Decision System Taskforce (4 December 2019). 
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See Lyria Bennett Moses and Edward Santow, ‘Accountability in 

the Age of Artificial Intelligence: A Right to Reasons’ (2020) 94(11) 
Australian Law Journal 829, 831; Jenna Burrell, ‘How the Machine 
‘Thinks’: Understanding Opacity in Machine Learning Algorithms’ 
(2016) 3(1) Big Data and Society 1; Monika Zalnieriute, Lyria Bennett 
Moses and George Williams, ‘The Rule of Law and Automation of 
Government Decision-making’ (2019) 82(3) Modern Law Review 425; 
Yee-Fui Ng, Maria O'Sullivan, Moira Paterson and Normann Witzleb 
‘Revitalising Public Law in a Technological Era: Rights, Transparency 
and Administrative Justice’ (2020) 43(3) UNSW Law Journal 1041. 

UK Information Commissioner's Office, Explaining Decisions Made 
with Al (20 May 2020). 

OECD, Recommendation of the Council on Artificial Intelligence, OECD/ 
Legal/0449 (adopted 22 May 2019). 

Australian Government, Department of Industry, Innovation and 
Science, Al Ethics Framework (November 2019). 

IEEE, Ethically Aligned Design: A Vision for Prioritizing Human Well- 
being with Autonomous and Intelligent Systems (Version 2 For Public 
Discussion, December 2017) 71. 

There is no general common law duty for decision makers to 
provide reasons, or explanations for decisions: Public Service 

Board of New South Wales v Osmond (1986) 159 CLR 656. The 
legislative review process provides clear avenues for review of 
decision making and a right to reasons: in Re Palmer and Minister 
for the Capital Territory (1978) 1 ALD 183 at 192, Fisher J (Deputy 
President), A N Hall (Senior Member) and C A Woodley (Member) 
said that obtaining reasons is ‘fundamental to the whole scheme 
of administrative review embodied in the [AAT] Act’. 

Administrative Appeals Tribunal Act 1975 (Cth) s 28. 

Administrative Decisions (Judicial Review) Act 1977 (Cth) s 13. This 
includes administrative decisions made under a Commonwealth 
Act, but not including decisions covered by s 28 of the Administrative 
Appeals Tribunal Act 1975 (Cth) or decisions described in Schedule 2 
of the Administrative Decisions (Judicial Review) Act 1977 (Cth). 

DP submissions: Information and Privacy Commission NSW, 19-21 
(arguing for reform of state information access laws to provide 
greater access to information held by governments around machine 
learning); K Weatherall, T Caetano, 18; S Hook, 9. 

DP submission: K Weatherall, T Caetano, 17, 18. 

Andrew Smith, ‘Using Artificial Intelligence and Algorithms’, Federal 
Trade Commission (online, 8 April 2020) <https://www.ftc.gov/news- 
events/blogs/business-blog/2020/04/using-artificial-intelligence- 
algorithms>. 

Sandra Wachter, Brent Mittelstadt and Luciano Floridi, ‘Why a Right 
to Explanation of Automated Decision-making Does Not Exist in the 
General Data Protection Regulation’ /nternational Data Privacy Law 
(2017) 7(2) 76. Compare with the opinion of Andrew D Selbst and 
Julia Powles, ‘Meaningful Information and the Right to Explanation’ 
(2017) 7(4) International Data Privacy Law 233. 

European Parliamentary Research Service, The Impact of the General 
Data Protection Regulation (GDPR) on Artificial Intelligence (15 June 
2020) 64-65. 

DP submissions: Aboriginal and Torres Strait Islander Legal Service 
Qld, 2; Access Now, 4; Al Now Institute, 7; Australian Council of 
Learned Academies, 4; Australian Industry Group, 5 (supported the 
concept of explainability in principle); Australian Lawyers Alliance, 1; 
Liberty Victoria, Australian Privacy Foundation, Queensland Council 
for Civil Liberties, Electronic Frontiers Australia, NSW Council for Civil 
Liberties, 7; Capgemini, 5 (recognised the ‘overarching obligation’ 
and need to provide explanations for Al-informed decisions, 
particularly where there is an adverse outcome for an individual); 
Castan Centre for Human Rights Law, 18; Commonwealth Bank 

of Australia, 4, 6 (offered qualified support, transparency through 
explanation ‘supported by the ongoing development of Al 
standards by the Australian Government); Community and Public 
Sector Union, 1; Consumer Policy Research Centre, 20 (offered 
broad support to Proposal 7, and the Commission's ‘commitment 
to greater accountability in this field’); Digital Rights Watch, 3; 
Element Al, 3; Ethics Matters Pty Ltd, 3; Financial Rights Legal 
Centre, 3 (provided qualified support for the proposal, noting 
support only if it applies to financial services decisions); Information 
and Privacy Commission NSW, 14; J Ng, 2; J Powles, W Bateman, 3 
(supporting the substance of Proposal 7, but considered them to 
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be underdeveloped); K Weatherall, T Caetano, 10; KPMG Australia, 
15; Law Council of Australia, 16; Maurice Blackburn Lawyers, 13; 
Microsoft, 6 (agreed in principle with the proposal to provide an 
explanation); NSW Bar Association, 7; NSW Young Lawyers, 11; 
Office of the Australian Information Commissioner, 5; Office of 
the Victorian Information Commissioner, 4; Sova Assessment, 5; 

R Walden, 6; QUT Digital Media Research Centre, 3; S Hook, A Cody, 
R Sappideen, Z Wang, J Juriandz, H Al-Alosi, L Spencer, E Seymour, 
S Kozlina, J Whelan, S Noakes, 10; University of Melbourne, 3; 
WiseLaw, 4; Australian Communications Consumer Action Network, 
13; B Auckram, 6. 

DP submissions: Aboriginal and Torres Strait Islander Legal 
Service Qld (Ltd), 3; Australian Lawyers Alliance, 2; Element Al, 3; 
Information and Privacy Commission NSW, 14; J Ng (who supports 
the inclusion in Australian legislation of a Data Protection Impact 
Assessment to support the ‘right of explanation’ on automated 
decisions); University of Melbourne, 5; Australian Communications 
Consumer Action Network, 13; Microsoft, 6 (agreeing in principle 
to legislation, but recommended considering ‘less prescriptive 
methods first given the time taken to pass legislation, such as 
internal reviews, explanatory notes and guidelines). 

DP submission: Microsoft, 6 (stated ‘detailed consideration’ is 
necessary to determine if new legislation is required, overly 
prescriptive legislation may not be feasible). 

DP submissions: K Weatherall, T Caetano, 9. 10; Amazon Web 
Services, 3; Capgemini, 6; Information and Privacy Commission 
NSW, 11; Consumer Policy Research Centre, 13 (highlighting the 
importance of understandability). 

DP submissions: K Weatherall, T Caetano, 10; Capgemini, 6. 

DP submissions: KPMG Australia, 16; Medical Insurance Group 
Australia, 4 (arguing that healthcare should be carved out from 
these discussions). 

Australian Human Rights Commission, Human Rights and Technology 
(Discussion Paper, December 2019) Proposal 8. 

DP submissions: Actuaries Institute, 3 (‘if adopted, seems 
reasonable to apply in any context, not just Al’); Liberty Victoria, 
Australian Privacy Foundation, Queensland Council for Civil 
Liberties, Electronic Frontiers Australia, NSW Council for Civil 
Liberties, 7; Ethics Matters Pty Ltd, 3; J Powles, W Bateman, 3 
(agreeing with the proposal in substance, but considered the 
proposals underdeveloped without the provision of clear remedial 
frameworks for breaches of the obligations to explain how Al 

was used); Law Council of Australia, 17; Office of the Victorian 
nformation Commissioner, 4; S Hook, 10; The Australia Institute’s 
Centre for Responsible Technology, 9; University of Melbourne, 5; 
WiseLaw, 5. 

DP submissions: Amazon Web Services, 3; Australian Academy 

of Science, 5; K Weatherall T Caetano, 11; Microsoft, 6; NSW Bar 
Association, 9; R Chalmers, 7; Telstra, 4. 

DP submissions: Amazon Web Services, 3; KPMG Australia, 22; 
Microsoft, 6; Digital Industry Group Inc, 10. 

DP submissions: Amazon Web Services, 3; Australian Academy of 
Science, 5; Telstra, 4. 

DP submissions: NSW Bar Association, 10; Pymetrics, 3. 

DP submissions: NSW Bar Association, 9; K Weatherall, T Caetano, 
11. 

DP submissions: Aboriginal and Torres Strait Islander Legal Service 
(Qld), 3; Liberty Victoria, Australian Privacy Foundation, Queensland 
Council for Civil Liberties, Electronic Frontiers Australia, NSW 
Council for Civil Liberties, 8; QUT Digital Media Research Centre, 3; 
nformation and Privacy Commission NSW, 15; Maurice Blackburn 
Lawyers, 3; NSW Young Lawyers, 3; S Hook, 10; WiseLaw, 5; 
Australian Lawyers Alliance, 5. 

DP submissions: Allens Hub for Technology, Law and Innovation, 
3; Amazon Web Services, 3; Commonwealth Bank of Australia, 

7; Kablamo Pty Ltd, 2; K Weatherall, T Caetano, 16 (considered 

it a possible approach, but noted the proposal, as written in 

the Discussion Paper, lacks nuance, and leaves many questions 
unanswered); Microsoft, 7; R Chalmers, 7; SAS Institute Australia, 9. 
DP submissions: Allens Hub for Technology, Law and Innovation, 4; 
Commonwealth Bank of Australia, 7. 

DP submission: Amazon Web Services, 3. 
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Brent Mittelstadt, Chris Russell and Sandra Wachter, ‘Explaining 
explanations in Al’, (Conference Paper, FAT*19, January 2019). 

DP submissions: Element Al, 3; Access Now, 4, 6; Australian Lawyers 
Alliance, 4; Castan Centre for Human Rights Law, 18; Maurice 
Blackburn Lawyers, 13; Law Council of Australia, 16; B Auckram, 6. 
UN Committee on the Elimination of Racial Discrimination, General 
Recommendation No 36: Preventing and Combating Racial Profiling by 
Law Enforcement Officials, CERD/C/GC/36 (24 November 2020) [31]. 
DP submissions: Information and Privacy Commission NSW, 14; 
Office of the Australian Information Commissioner, 5; Australian 
Lawyers Alliance, 4; Al Now Institute, 7; Aboriginal and Torres Strait 
Islander Legal Service (Qld), 2; Castan Centre for Human Rights Law, 
18; Maurice Blackburn Lawyers, 13; Law Council of Australia, 16; The 
Australia Institute’s Centre for Responsible Technology, 8; University 
of Melbourne, 5. 

DP submissions: R Chalmers, 6; Capgemini, 5; Telstra, 4; 

K Weatherall, T Caetano, 15. 

DP submissions: Information and Privacy Commission NSW, 14; SAS 
Institute Australia, 8; K Weatherall, T Caetano, 10. 

DP submissions: Al Now Institute, 7; R Chalmers, 6; Capgemini, 6. 
High-Level Expert Group on Artificial Intelligence, Ethics Guidelines 
for Trustworthy Al (European Commission, 8 April 2019) 18. 

DP submissions: Liberty Victoria, Australian Privacy Foundation, 
Queensland Council for Civil Liberties, Electronic Frontiers Australia, 
NSW Council for Civil Liberties, 9; Castan Centre for Human Rights 
Law, 23 (in favour in principle, but queried whether it would have 
the practical effect of increasing decision-making transparency); 
Commonwealth Bank of Australia, 6 (if it applies only where a 
person would otherwise have been entitled to an explanation); NSW 
Council for Civil Liberties, 6; Information and Privacy Commission 
NSW, 21; Maurice Blackburn Lawyers, 4; WiseLaw, 5. 

DP submissions: Commonwealth Bank of Australia, 5; Law Council 
of Australia, 16; Digital Rights Watch, 3; NSW Young Lawyers, 12. 

DP submissions: Commonwealth Bank of Australia, 6; K Weatherall, 
T Caetano, 11, 15; Microsoft, 6; NSW Young Lawyers, 12. 

DP submissions: Al Now Institute, 7; Maurice Blackburn Lawyers, 
13-16. 

DP submissions: Information and Privacy Commission NSW, 19-21 
(arguing for reform of state information access laws to provide 
greater access to information held by governments around machine 
learning); K Weatherall, T Caetano, 18; S Hook, 9. 

DP submissions: Amazon Web Services, 4; Castan Centre for Human 
Rights Law, 23; Commonwealth Bank of Australia, 7; Digital Industry 
Group Inc, 7; Microsoft, 7; Telstra, 4. 

DP submissions: Amazon Web Services, 4; Allens Hub for 
Technology, Law and Innovation, 4; R Chalmers, 8. 

DP submissions: Al Now Institute, 8; Allens Hub for Technology, Law 
and Innovation, 5. 

See, for example, DP submission: Castan Centre for Human Rights 
Law, 23. 

DP submissions: Al Now Institute, 8; Actuaries Institute, 5. 

See Monika Zalnieriute, Lyria Bennett Moses and George Williams, 
‘The Rule of Law and Automation of Government Decision-making’ 
(2019) 82(3) Modern Law Review 425. 

Ansett Transport Industries (Operations) Pty Ltd v Wraith (1983) 48 

ALR 500, 507 (Woodward J). 

Burns v Australian National University (1982) 40 ALR 707; 61 FLR 

76, 85; Secretary, Dept of Foreign Affairs and Trade v Boswell (1992) 
108 ALR 77, 88. 

Re Minister for Immigration and Multicultural and Indigenous Affairs; 
Ex parte Palme (2003) 216 CLR 212, 242 (Kirby J). 

96% respondents expect Al to be regulated and 66% expect 
government oversight of Al: Nicole Gillespie, Steve Lockey and 
Caitlin Curtis, Trust in Artificial Intelligence: Australian Insights 
(University of Queensland and KPMG Australia, October 2020) 

3; ‘The majority of the public agrees with the need for industry 
guidelines and/or new legislation to address all possible harms 
associated with Al’: Neil Selwyn, Beatriz Gallo Cordoba, Mark 
Andrejevic and Liz Campbell, AI for Social Good? Australian Public 
Attitudes Toward Al and Society (Monash University, August 2020) 

9; Between 41% and 48% of participants would have ‘a lot more 
trust’ and between 30% and 33% would have ‘a little more trust’ in 
an automated decision if any of the oversight measures including 
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human checks, limitations on personal information sharing within 
the government and stronger laws to protect their rights were 
implemented: Essential Research, The Essential Report—Human 
Rights Commission (29 July 2020). 

Ombudsman Act 1976 (Cth) s 15. 

Toby Walsh, Neil Levy, Genevieve Bell, Anthony Elliott, James 
Maclaurin, Iven Mareels and Fiona Wood, The Effective and Ethical 
Development of Artificial Intelligence: An Opportunity to Improve our 
Wellbeing (Report for the Australian Council of Learned Academies, 
July 2019) 124. 

Commonwealth Ombudsman, Centrelink’s Automated Debt Raising 
and Recovery System: A Report About the Department of Human 
Services’ Online Compliance Intervention System for Debt Raising and 
Recovery (April 2017). 

67% respondents considered it ‘very important’ and 21% considered 
it ‘somewhat important’: Essential Research, The Essential Report— 
Human Rights Commission (29 July 2020). 

Monika Zalnieriute, Lyria Bennett Moses and George Williams, ‘The 
Rule of Law and Automation of Government Decision-making’ (2019) 
82(3) Modern Law Review 425. 

Trivago NV v Australian Competition and Consumer Commission [2020] 
FCAFC 185, [64]. 

UK Information Commissioner's Office, Explaining Decisions Made 
with Al (20 May 2020); European Parliamentary Research Service, The 
Impact of the General Data Protection Regulation (GDPR) on Artificial 
Intelligence (15 June 2020) 54. Pointed out also in DP submissions, 
see, for example: SAS Institute Australia, 8; Office of the Victorian 
Information Commissioner, 4; K Weatherall, T Caetano, 10. 
European Parliamentary Research Service, The Impact of the General 
Data Protection Regulation (GDPR) on Artificial Intelligence (15 June 
2020) 54. 

For discussion of the range of information that may be required to 
explain an Al-informed decision see DP submissions: K Weatherall, 
T Caetano, 17; SAS Institute Australia, 8. See also UK Information 
Commissioner's Office, Explaining Decisions Made with Al (20 May 
2020); Office of the Victorian Information Commissioner, Closer to 
the Machine: Technical, Social and Legal Aspects of Al (August 2019) 56. 
All-Party Parliamentary Group on Artificial Intelligence, Ethics and 
Legal in Al: Decision Making and Moral Issues (March 2017) 6; Robert 
French AC, ‘Rationality and Reason in Administrative Law - Would 

a Roll of the Dice be Just as Good?’ (Australian Academy of Law 
Lecture, 29 November 2017) <https://www.fedcourt.gov.au/digital- 
law-library/seminars/r-french-20171120> 4. 

68% survey respondents considered this ‘very important’ and 19% 
‘somewhat important’: Essential Research, The Essential Report— 
Human Rights Commission (29 July 2020). 

See, for example, Data Protection Act 2018 (UK) s 14(4). 

See, for example, Jake Goldenfein ‘Algorithmic Transparency 

and Decision-making Accountability’, in Office of the Victorian 
Information Commissioner, Closer to the Machine: Technical, Social 
and Legal Aspects of Al (August 2019) 56. 

DP submissions: R Chalmers, 8; KPMG Australia, 15; Al Now 
Institute, 10. 

DP submissions: Aboriginal and Torres Strait Islander Legal Service 
(Qld), 3; Allens Hub for Technology, Law and Innovation, 6; Castan 
Centre for Human Rights Law, 22, 24 Liberty Victoria, Australian 
Privacy Foundation, Queensland Council for Civil Liberties, Electronic 
Frontiers Australia, NSW Council for Civil Liberties, 9; Ethics Matters 
Pty Ltd, 3; SAS Institute Australia, 11. 

DP submission: Al Now Institute, 9, 10; Allens Hub for Technology, 
Law and Innovation, 4. 

DP submissions: Aboriginal and Torres Strait Islander Legal Service 
(Qld), 3; Allens Hub for Technology, Law and Innovation, 6; Castan 
Centre for Human Rights Law, 22; Ethics Matters Pty Ltd, 3. 

DP submission: Allens Hub for Technology, Law and Innovation, 6. 
DP submissions: Commonwealth Bank of Australia, 8; Information 
and Privacy Commission NSW, 21; Actuaries Institute, 4. 
Administrative Decisions (Judicial Review) Act 1977 (Cth) s 5. 
Administrative Decisions (Judicial Review) Act 1977 (Cth) s 16. 

See Lyria Bennett Moses and Edward Santow, ‘Accountability in 

the Age of Artificial Intelligence: A Right to Reasons’ (2020) 94(11) 
Australian Law Journal 829, 833. 

Pintarich v Deputy Commissioner of Taxation [2018] FCAFC 79. 
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Pintarich v Deputy Commissioner of Taxation [2018] FCAFC 79, 

[140]; approving the decision of Finn J in Semunigus v Minister for 
Immigration and Multicultural Affairs [1999] FCA 422, [19]; affirmed in 
Seamunigus v Minister for immigration and Multicultural Affairs [2000] 
FCA 240. 

Pintarich v Deputy Commissioner of Taxation [2018] FCAFC 79, [46], 
[49]. Compare Gramar Pty Ltd (t/as Valley View Nursing Home) v 
Workcover Corporation of South Australia [2011] SASC 237; the 
Supreme Court of South Australia considered the plaintiff's 
argument that the imposition of a supplementary levy, calculated 
by an automated computer-generated process, was not a decision 
by the delegated authority under the legislation in question and 
therefore was an invalid decision that should be set aside. Justice 
Kelly determined that as the automated process was based on 

a formula adopted and maintained for the purposes of the levy 
scheme some years prior: ‘Effectively therefore the rate was arrived 
at by the defendant in accordance with its policies, even though 
the precise calculation was performed by a computer, using the 
formula’: [27]-[B4]. 

All-Party Parliamentary Group on Artificial Intelligence, Ethics and 
Legal in Al: Decision Making and Moral Issues (March 2017) 6; Robert 
French AC, ‘Rationality and Reason in Administrative Law - Would 

a Roll of the Dice be Just as Good?’ (Australian Academy of Law 
Lecture, 29 November 2017) <https://www.fedcourt.gov.au/digital- 
law-library/seminars/r-french-20171120> 4. 

68% survey respondents considered this ‘very important’ and 19% 
‘somewhat important’: Essential Research, The Essential Report— 
Human Rights Commission (29 July 2020). 

Government of Canada, Directive on Automated Decision-Making 
(February 2019) <https://www.tbs-sct.gc.ca/pol/doc-eng. 
aspx?id=32592> Part 6.39. 

Georgina Ibarra, Meena Tharmarajah and David Douglas, ‘Designing 
Trustworthy Machine Learning Systems’, CS/RO’s Data67 (online, 

5 November 2020) <https://algorithm.data61 .csiro.au/designing- 
trustworthy-machine-learning-systems/>. 

European Parliamentary Research Service, The Impact of the General 
Data Protection Regulation (GDPR) on Artificial Intelligence (15 June 
2020) 22. 

Commonwealth Ombudsman, Automated Decision-making Better 
Practice Guide (2019). 

See, for example, Administrative Appeals Tribunal Act 1975 (Cth) s 43. 
See UN Office of the High Commissioner for Human Rights, 
Addressing Business Model Related Human Rights Risks: A B-Tech 
Foundational Paper (2020) <https://www.ohchr.org/Documents/ 
Issues/Business/B-Tech/B_Tech_Foundational_Paper.pdf>. 

UDHR art 8; /CCPR art 2(3); ICERD art 6; CAT art 14; CRC art 39. 

UN Human Rights Committee, General Comment No 31 [80]: The 
Nature of the General Legal Obligation Imposed on States Parties to the 
Covenant, 80" sess, UN Doc CCPR/C/21/Rev/1/Add/13 (26 May 2004, 
adopted 29 March 2004). 

While the legal dividing line between government and non- 
government decisions is generally obvious, there are some 
situations where it is less clear (for example, where a non- 
government entity makes a decision on behalf of the government). 
See, generally, NEAT Domestic Trading Pty Ltd v AWB (2003) 216 CLR 
277; 

Human Rights Act 2004 (ACT) s 40(1)(g); Charter of Human Rights and 
Responsibilities Act 2006 (Vic) s 4(1)(c); Human Rights Act 2019 (Qld) 

ss 9-10. 

Corporations Act 2001 (Cth) s 180. 

Corporations Act 2001 (Cth) s 180. 

Samar Ashour, ‘Artificial Intelligence in the Boardroom: An Outright 
Exposure to Directorial Liability’, Oxford Business Law Blog (online, 
12 October 2020) <https://www.law.ox.ac.uk/business-law-blog/ 
blog/2020/10/artificial-intelligence-boardroom-outright-exposure- 
directorial>. 

This is explored further in Chapter 9. 

Australian Government, Department of Industry, Innovation and 
Science, Al Ethics Framework (November 2019). 

Australian Government, Department of Industry, Science, Energy 
and Resources, An Al Action Plan for all Australians: A Call for Views - 
Discussion Paper (2020) 7. 
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DP submissions: Element Al, 3; Microsoft, 5; Al Now Institute, 4; 
Australian Academy of Science, 3; Wiselaw, 3; Maurice Blackburn 
Lawyers, 12; Law Council of Australia, 14. 

DP Submission: Microsoft, 5. 

DP submission: Capgemini, 7. 

DP submissions: Microsoft, 5; Australian Academy of Science, 3. 
Australian Government, Department of Industry, Innovation and 
Science, Al Ethics Framework (November 2019). 

Nicole Gillespie, Steve Lockey and Caitlin Curtis, Trust in Artificial 
Intelligence: Australian Insights (University of Queensland and KPMG 
Australia, October 2020) 31. 

UN Office of the High Commissioner for Human Rights, Guiding 
Principles on Business and Human Rights: Implementing the United 
Nations “Protect, Respect and Remedy” Framework (HR/PUB/1 1/04, 
2011) Principle 21, 23. 

David Kaye, Report of the Special Rapporteur on the Promotion and 
Protection of the Right to Freedom of Opinion and Expression, UN Doc 
A/73/348 (29 August 2018) 21. 

Council of Europe Commissioner for Human Rights, Unboxing 
Artificial Intelligence: 10 Steps to Protect Human Rights (May 2019) 9. 
Sale of Automated Employment Decision Tools Bill, Intro 1894-2020, 
New York City (27 February 2020). 

Australian Securities and Investment Commission (ASIC), Responsible 
Lending Disclosure Obligations: Overview for Credit Licensees and 
Representatives (Information Sheet 146, October 2020). 

Karen Yeung, Rapporteur, A Study of the Implications of Advanced 
Digital Technologies (Including Al Systems) for the Concept of 
Responsibility Within a Human Rights Framework, Council of Europe 
MSI-AUT(2018)05 (9 November 2018) 74. Product safety and 
questions of liability for Al technologies are currently under 
consideration by the EU: see European Commission, Report on the 
Safety and Liability Implications of Artificial Intelligence, the Internet of 
Things and Robotics (COM/2020/64, 19 February 2020) 74. 

IP submissions: R McLay, 9.1; University of Technology Sydney, 

34; LexisNexis, 16; Royal Australian and New Zealand College of 
Radiologists, 5; Intopia, 6, 7; P Henman, 6; NSW Young Lawyers 
Communications, Entertainment and Technology Law Committee, 
11; N Suzor, K Weatherall, A Daly, A Vromen, M Mann, 3. WP 
submissions: Law Council of Australia, 2; Montreal Al Ethics Institute, 
7; Royal Australian and New Zealand College of Radiologists, 3; R 
Clarke, 17; Microsoft, 5. See also Toby Walsh, Neil Levy, Genevieve 
Bell, Anthony Elliott, James Maclaurin, Iven Mareels and Fiona 
Wood, The Effective and Ethical Development of Artificial Intelligence: 
An Opportunity to Improve our Wellbeing (Report for the Australian 
Council of Learned Academies, July 2019) 137. 

Australian Human Rights Commission, Human Rights and Technology 
(Discussion Paper, December 2019) Proposal 10. 

DP submissions: Al Now Institute, 11; Liberty Victoria, Australian 
Privacy Foundation, Queensland Council for Civil Liberties, Electronic 
Frontiers Australia, NSW Council for Civil Liberties, 9; Castan Centre 
for Human Rights Law, 22 (only ‘in instances where an Al system 
could adapt and generate an outcome not intended or reasonably 
foreseen by the user of the Al system’); Digital Rights Watch, 3; 
Information and Privacy Commission NSW, 14; Law Council of 
Australia, 17; Maurice Blackburn Lawyers, 14; NSW Bar Association, 
10; Office of the Victorian Information Commissioner, 4; University 
of Technology Sydney, 23; Victoria Legal Aid, 4; WiseLaw, 5; The 
Australia Institute’s Centre for Responsible Technology, 8; QUT 
Digital Media Research Centre, 4; NSW Young Lawyers, 4; Amazon 
Web Services, 4. 

DP submissions: Amazon Web Services, 4; Australian Industry 
Group, 6; Digital Industry Group Inc, 10; J Powles, W Bateman, 3; 
Kablamo Pty Ltd, 2; Microsoft, 3; SAS Institute Australia, 10; Medical 
nsurance Group Australia, 3. 

DP submissions: Microsoft, 3, J Powles, W Bateman, 3; 
Commonwealth Bank of Australia, 7; Law Council of Australia, 18. 
DP submissions: Al Now Institute, 11; Liberty Victoria, Australian 
Privacy Foundation, Queensland Council for Civil Liberties, Electronic 
Frontiers Australia, NSW Council for Civil Liberties, 9; 1zerobzero, 2; 
SAS Institute Australia, 10; Telstra, 4; Australian Industry Group, 6; 
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Castan Centre for Human Rights Law, 22; Law Council of Australia, 
18. 

DP submissions: Amazon Web Services, 4; Australian Industry 
Group, 6; KPMG Australia, 16; University of Sydney, 23; Digital 
Industry Group Inc, 10. 

See, for example, the Banking Executive Accountability Regime 
(BEAR), Banking Act 1959 (Cth) pt IIAA. 

Competition and Consumer Act 2010 (Cth) s 155 confers powers 

on the ACCC to obtain information, documents and evidence in 
relation to specified matters that constitute or may constitute a 
contravention of the Act and other legislative provisions. See ACCC, 
ACCC Guidelines—Use of Section 155 Powers (June 2020). 

Australian Human Rights Commission, Human Rights and Technology 
(Discussion Paper, December 2019) Part B. 

DP submissions: Amazon Web Services, 3; Digital Industry Group 
Inc, 9; Kablamo Pty Ltd, 2; Medical Insurance Group Australia, 3 
(argued against a positive duty of explanation being applied to 
healthcare, where there are already existing obligations around 
disclosure); Telstra, 4, (recognising the importance of explainability 
and accountability in building public trust in Al-informed decision 
making, but strongly opposed to a legislated explainability 
requirement). 

DP submissions: Australian Industry Group, 5; Telstra, 4; Pymetrics, 
3; Microsoft, 7; W Small, 1; DP Consultation roundtables: 4, 15. 

DP submissions: Capgemini, 6; Commonwealth Bank of Australia, 6; 
NSW Bar Association, 7. 

DP submission: Allens Hub for Technology and Law, 4. 

DP submission: University of Technology Sydney, 16. 

DP submissions: Allens Hub for Technology, Law and Innovation, 

4; Microsoft, 7; NSW Bar Association, 8; Telstra, 4; University of 
Technology Sydney, 21; Commonwealth Bank of Australia, 4. 

DP submissions: K Weatherall, T Caetano, 9. 10; Amazon Web 
Services, 3; Capgemini, 6; Information and Privacy Commission 
NSW, 14; Consumer Policy Research Centre, 13 (highlighting the 
importance of understandability). 

DP submissions: Victoria Legal Aid, 3; Al Now Institute, 6; NSW Bar 
Association, 9. 

The Law Council of Australia, for example, argued a public interest 
steward could be offered as an alternative, or an addition, to public 
disclosure: DP submission: Law Council of Australia, 16. See also 
DP submissions: NSW Bar Association, 8; Office of the Australian 
Information Commissioner, 5. 

DP submissions: K Weatherall, T Caetano, 10; Capgemini, 6. 

DP submissions: Aboriginal and Torres Strait Islander Legal Service 
(Qld), 3; Liberty Victoria, Australian Privacy Foundation, Queensland 
Council for Civil Liberties, Electronic Frontiers Australia, NSW 
Council for Civil Liberties, 8; QUT Digital Media Research Centre, 3; 
Information and Privacy Commission NSW, 15; Maurice Blackburn 
Lawyers, 3; NSW Young Lawyers, 3; S Hook, 10; WiseLaw, 5; 
Australian Lawyers Alliance, 5. 

DP submission: Financial Rights Legal Centre, 32. 

DP submissions: Allens Hub for Technology, Law and Innovation, 

3; Amazon Web Services, 3; Commonwealth Bank of Australia, 

7; Kablamo Pty Ltd, 2; K Weatherall, T Caetano, 16 (considered 

it a possible approach, but noted the proposal, as written in 

the Discussion Paper, lacks nuance, and leaves many questions 
unanswered); Microsoft, 7; R Chalmers, 7; SAS Institute Australia, 9. 
DP submissions: Allens Hub for Technology, Law and Innovation, 4; 
KPMG Australia, 22. 

DP submissions: Allens Hub for Technology, Law and Innovation, 4; 
Commonwealth Bank of Australia, 7. 

DP submission: Amazon Web Services, 3. 

See, for example, written evidence submitted by The Alan Turing 
Institute, to the UK House of Commons Science and Technology 
Committee Inquiry: Algorithms in Decision-making (28 February 
2017). 

Australian Human Rights Commission, Human Rights and Technology 
(Discussion Paper, December 2019) Proposal 9. 
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DP submissions: Allens Hub for Technology, Law and Innovation, 
3; Australian Council of Learned Academies, 5; Australian Industry 
Group, 5; Commonwealth Bank of Australia, 6; Digital Rights Watch, 
4; Digital Industry Group Inc, 10; Ethics Matters Pty Ltd, 3; QUT 
Digital Media Research Centre, 3; KPMG Australia, 22; Office of the 
Victorian Information Commissioner, 4; WiseLaw, 5. 

DP submissions: Allens Hub for Technology, Law and Innovation, 3; 
Australian Council of Learned Academies, 5; R Chalmers, 7. 

See Communications Alliance Ltd, Industry Code C268:2019 
Telecommunications Consumer Protections Code (July 2019) 45. 

DP Submissions: K Weatherall, T Caetano, 13; Financial Rights Legal 
Centre, 3; SafeGround Inc. 

Privacy Act 1988 (Cth) s 21P, for example, provides that where a 
credit provider refuses an application for consumer credit, and the 
refusal is based wholly or partly on credit eligibility information, the 
credit provider must provide specific information to the applicant, in 
writing, regarding the refusal. 

Competition and Consumer Act 2010 (Cth) s 155 confers powers 

on the ACCC to obtain information, documents and evidence in 
relation to specified matters that constitute or may constitute a 
contravention of the Act and other legislative provisions; ACCC, 
ACCC Guidelines—Use of Section 155 Powers (June 2020). 

Australian Securities and Investment Commission Act 2001 (Cth) gives 
ASIC a range of compulsory information-gathering powers to 
compel a person or entity to provide documents and information, 
and attend an examination: ASIC, ASIC's Compulsory Information- 
gathering Powers (Information Sheet 145, March 2020). 

Privacy Act 1988 (Cth) ss 44, 45. 

See, for example, Aleksandra Mojsilovic, ‘Introducing Al 
Explainability 360’, IBM (online, 8 August 2019) <https://www.ibm. 
com/blogs/research/2019/08/ai-explainability-360/>. 

Executive Office of the President of the United States, Maintaining 
American Leadership in Artificial Intelligence, Executive Order 13859 
of 11 February 2019 (14 February 2019). 

Australian Government, Australian Research Council, 2020 ARC 
Centre of Excellence for Automated Decision-Making and Society 

(7 January 2021) <https://www.arc.gov.au/2020-arc-centre- 
excellence-automated-decision-making-and-society>. 

UN Office of the High Commissioner for Human Rights, Who Will be 
Accountable? Human Rights and the Post-2015 Development Agenda 
(HR Pub/13/1, 2013) 10. 

Australian Government, Administrative Review Council, 
Administrative Accountability in Business Areas Subject to Complex and 
Specific Regulation (2008) 5. 

Australian Communications and Media Authority (ACMA), Optimal 
Conditions for Effective Self- and Co-regulatory Arrangements 
(Occasional Paper, September 2011) 4. 

ACMA, Optimal Conditions for Effective Self- and Co-regulatory 
Arrangements (Occasional Paper, September 2011) 5. 

ACMA, Optimal Conditions for Effective Self- and Co-regulatory 
Arrangements (Occasional Paper, September 2011) 12. 

ACMA, Artificial Intelligence in Communications and Media (Occasional 
Paper, July 2020) 26. 

DP submissions: Element Al, 3; KPMG Australia, 2, 8; Microsoft, 

1; Digital Industry Group Inc, 8; PwC, 2; WiseLaw, 1; Telstra, 1; 
Responsible Technology Australia, 4; IP submissions: Chartered 
Accountants Australia and New Zealand, 23, 3. 

DP submissions: Australian Services Union, 4; Information and 
Privacy Commission NSW, 1; R Chalmers, 1; Pymetrics, 2; Public 
Interest Advocacy Centre, 1;University of Technology Sydney, 54; 
Digital Gap Initiative, 12; Responsible Technology Australia, 4; Law 
Council of Australia, 5. IP submissions: WebKeylT, 7(b); Confidential 
submission 18, response to question 7(b); Confidential submission 
2,7(b); S Ramesh 7(b); S Murphy, 7(b); P Henman, 8; Castan Centre 
for Human Rights Law, 19, 28; EMR Australia, 14; University of 
Technology Sydney, 61; Castan Centre for Human Rights Law, 20, 28. 
Harvard University’s Berkman Klein Center for Internet & Society 
at Harvard University has conducted thematic mapping of 47 
ethical frameworks: Jessica Fjeld, Nele Achten, Hannah Hilligoss, 
Adam Christopher Nagy and Madhulika Srikumar, Principled 
Artificial Intelligence: Mapping Consensus in Ethical and Rights-Based 
Approaches to Principles for Al (Berkman Klein Center, 15 January 
2020). 
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See, for example, OECD, OECD Principles on Al (9 June 2019) <https:// 
www.oecd.org/going-digital/ai/principles/>. 

Australian Government, Department of Industry, Innovation and 
Science, Al Ethics Framework (November 2019). 

Microsoft, Microsoft Al Principles <https://www.microsoft.com/en-us/ 
ai/responsible-ai?activetab=pivot1 %3aprimaryr6é>. 

Australian Human Rights Commission, Human Rights and Technology 
(Discussion Paper, December 2019) 52-54. 

Australian Government, Department of Industry, Innovation and 
Science, Al Ethics Framework (November 2019). 

Australian Government, Department of Industry, Science, Energy 
and Resources, Towards an Artificial Intelligence Ethics Framework 

(7 November 2019) <https://www.industry.gov.au/news/towards-an- 
artificial-intelligence-ethics-framework>. 

DP submissions: PwC, 1; R Chalmers, 2; Australian Red Cross, 

2; Microsoft, 2; Digital Industry Group Inc, 3; Information and 
Privacy Commission NSW, 5; Access Now, 2; Office of the eSafety 
Commissioner, 3. 

See, for example, IP submissions: University of Melbourne, 10; 
Digital Gap Initiative, 13; K Mathews-Hunt, 222; Australian Privacy 
Foundation, Queensland Council for Civil Liberties, Electronic 
Frontiers Australia, 29; University of Technology Sydney, 20. These 
views were also reflected in IP and DP Roundtable consultations. 
The following DP submissions discussed issues relating to ethical 
frameworks: Liberty Victoria, Australian Privacy Foundation, 
Queensland Council for Civil Liberties, Electronic Frontiers Australia, 
NSW Council for Civil Liberties, 3; Digital Industry Group Inc, 8; 
Access Now, 2; Australian Services Union, 4. See also IP submission: 
Australian Privacy Foundation, Electronic Frontiers Australia, 
Queensland Council for Civil Liberties; Castan Centre or Human 
Rights Law, 6, 7. 
DP submissions: K Weatherall, T Caetano, 1; J Powles, W Bateman. 
DP submissions: K Weatherall, T Caetano, 1; Australian Academy of 
Science, 3; KPMG Australia, 19; Consumer Policy Research Centre, 9; 
Microsoft, 2; Australian Services Union, 4. 

DP submissions: Law Council of Australia, 8; National Health and 
Medical Research Council, 1; Commonwealth Bank of Australia, 

3; Office of the Victorian Information Commissioner, 2; University 
of Technology Sydney, 32; Ethics Matters Pty Ltd, 1; WiseLaw, 1; 
Maurice Blackburn Lawyers, 8; Australian Red Cross, 2; PwC, 2. 

DP submissions: Office of the Victorian Information Commissioner, 
2; Chartered Accountants Australia and New Zealand, 1. 

DP submissions: Law Council of Australia, 8-9; Al Asia Pacific Forum, 
2; D Svantesson, 2-3. 

Jessica Morley, Luciano Floridi, Libby Kinsey and Anat Elhalal, ‘From 
What to How: An Initial Review of Publicly Available Al Ethics Tools, 
Methods and Research to Translate Principles into Practices’ (2020) 
26 Science and Engineering Ethics 2141; Anna Jobin, Marcello lenca 
and Effy Vayena, ‘Artificial Intelligence: The Global Landscape of 
Ethics Guidelines’ (2019) 1 Nature Machine Intelligence 389; Ben 
Wagner, ‘Ethics as an Escape From Regulation. From Ethics-washing 
to Ethics-shopping?’ in Emre Bayamlio§glu, Irina Baraliuc, Liisa 
Janssens and Mireille Hildebrandt (eds) Being Profiled: Cogitas Ergo 
Sum (Amsterdam University Press, 2018) 84-88; Ray Eitel-Porter 
‘Beyond the Promise: Implementing Ethical Al’ (2020) 1 Al Ethics 73. 
See, for example, Karen Yeung, Andrew Howes and Ganna 
Pogrebna, ‘Al Governance by Human Rights-Centred Design, 
Deliberation and Oversight: An End to Ethics Washing’ in Markus D 
Dubber, Frank Pasquale and Sunit Das (eds), The Oxford Handbook 
of Al Ethics (Oxford University Press, 2020) 77; Evgeni Aizenberg and 
Jeroen van den Hoven, ‘Designing for Human Rights and Al’ [2020] 
(July-December) Big Data & Society 1. 

See, for example, Luke Stark and Anna Lauren Hoffman, ‘Data is the 
New What? Popular Metaphors and Professional Ethics in Emerging 
Data Culture’ (2019) 1(1) Journal of Culture Analytics 1, 17. 

Eileen Donahoe and Megan MacDuffee Metzger, ‘Artificial 
Intelligence and Human Rights’ (2019) 30(2) Journal of Democracy 
115, 121. 

UN Office of the High Commissioner for Human Rights, “Smart 
Mix" of Measures Needed to Regulate New Technologies - Bachelet’ 
(Media Release, 24 April 2019). 
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Jessica Fjeld, Nele Achten, Hannah Hilligoss, Adam Christopher Nagy 
and Madhulika Srikumar, Principled Artificial Intelligence: Mapping 
Consensus in Ethical and Rights-Based Approaches to Principles for Al 
(Berkman Klein Center, 15 January 2020) 66. 

See Andrew McNamara, Justin Smith and Emerson Murphy-Hill, 
‘Does ACM's Code of Ethics Change Ethical Decision Making 

in Software Development?’ (Conference Paper, 26 ACM Joint 
Meeting, 2018); James Arvanitakis, ‘What are Tech Companies Doing 
About Ethical Use of Data? Not Much’ The Conversation (online, 

28 November 2018) <https://theconversation.com/what-are-tech- 
companies-doing-about-ethical-use-of-data-not-much-104845>. 
The Gradient Institute, Whose Ethics? (25 March 2019) <https:// 
gradientinstitute.org/blog/4/>. 

For discussion on the ongoing monitoring role of the recommended 
Al Safety Commissioner, see Section 10.3. 

See Karen Yeung, noting the importance of both judicial and 
non-judicial measures to support accountable use of Al, including 
by considering a ‘regulation by design’ process: Karen Yeung, 
Rapporteur, A Study of the Implications of Advanced Digital 
Technologies (Including Al Systems) for the Concept of Responsibility 
Within a Human Rights Framework, Council of Europe MSI-AUT (2018) 
05 (9 November 2018) 64-65. See also Karen Yeung, Andrew Howes 
and Ganna Pogrebna, ‘Al Governance by Human Rights-Centred 
Design, Deliberation and Oversight: An End to Ethics Washing’ in 
Markus D Dubber, Frank Pasquale and Sunit Das (eds), The Oxford 
Handbook of Al Ethics (Oxford University Press, 2020) 77. 

Karen Yeung, Andrew Howes and Ganna Pogrebna, ‘Al Governance 
by Human Rights-Centred Design, Deliberation and Oversight: An 
End to Ethics Washing’ in Markus D Dubber, Frank Pasquale and 
Sunit Das (eds), The Oxford Handbook of Al Ethics (Oxford University 
Press, 2020) 77. 

DP submissions: Confidential submission 16, 1; Office of the 
Australian Information Commissioner, 5; R Chalmers, 14. 

IP submission: Office of the Australian Information Commissioner, 
4. DP submission: Information and Privacy Commission NSW, 17. 
Australian Government, eSafety Commissioner, Safety by Design 
<https://www.esafety.gov.au/about-us/safety-by-design>; DP 
submissions: Confidential submission 16, 1; Digital Industry Group 
Inc, 1. 

DP submissions: Information and Privacy Commission NSW, 16; 
Ethics Matters Pty Ltd, 4; Microsoft, 6; R Walden, 6; Wiselaw, 7; 
University of Melbourne, 5; R Chalmers, 9; SAS Institute Australia, 
12; KPMG Australia, 24; SafeGround Inc; Pymetrics, 4; Australian 
Academy of Science, 6; Consumer Policy Research Centre, 5; 
University of Technology Sydney, 32; Z Milosevic, 1; J Ng, 5; 
Australian Council of Learned Academies, 4; Centre for Inclusive 
Design, 1. DP Consultation roundtable: 7. 

DP submissions: WiseLaw, 7; University of Melbourne, 5; 

R Chalmers, 9; KPMG Australia, 24. 

DP submission: SAS Institute Australia, 12. 

DP submissions: Information and Privacy Commissioner NSW,15; 
Pymetrics, 4; KPMG Australia, 24. 

See UN Office of the High Commissioner for Human Rights, Guiding 
Principles on Business and Human Rights: Implementing the United 
Nations “Protect, Respect and Remedy” Framework (HR/PUB/1 1/04, 
2011) ch Il. 

Standards Australia, Standardisation Guide 003: Standards and Other 
Publications (July 2019) 3. 

See, for example, OECD, OECD Principles on Al (9 June 2019) <https:// 
www.oecd.org/going-digital/ai/principles/>. 

European Commission, Report on the Safety and Liability Implications 
of Artificial Intelligence, the Internet of Things and Robotics 
(COM/2020/64, 19 February 2020). 

Australian Government, Department of Industry, Science, Energy 
and Resources, An Al Action Plan for all Australians: A Call for Views - 
Discussion Paper (2020) 13. 

Australian Government, Department of Industry, Science, Energy 
and Resources, An Al Action Plan for all Australians: A Call for Views - 
Discussion Paper (2020) 7. 

Standards Australia, Final Report: An Artificial Intelligence Standards 
Roadmap: Making Australia’s Voice Heard (Final Report, March 2020). 
Australian Government, Department of Industry, Innovation and 
Science, Al Ethics Framework (November 2019). 
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DP submissions: Information and Privacy Commission NSW, 15; 
Element AI, 4; SAS Institute Australia, 13; Ethics Matters Pty Ltd, 4; 
R Walden, 6; WiseLaw, 7. DP Consultation roundtable: 7. 

DP submissions: Element Al, 5; SAS Institute Australia, 13; 
WiseLaw, 7. 

General Data Protection Regulation art 42. 

See IP submissions: Deloitte, 6; Google, 2. DP submission: SAS 
Institute Australia, 13. 

For discussion on a potential Turing Stamp see IP submissions: 

E Fries; University of Melbourne, 10; Castan Centre for Human 
Rights Law, 82; Deloitte, 6. 

DP submission: Office of the Australian Information Commissioner, 
6. 

DP submission: Actuaries Institute, 5, 6. 

DP submission: Digital Industry Group Inc, 11. 

See DP submissions: Information and Privacy Commission NSW, 15; 
SAS Institute Australia, 13; Actuaries Institute, 5; KPMG Australia, 24; 
Pymetrics, 4; Australian Academy of Science, 7; J Crock, 26; Wiselaw, 
6; Australian Council of Learned Academies, 4. 

See Mark Hodge and Dunstan Allison-Hope, Artificial Intelligence: 

A Rights-Based Blueprint for Business (Working Paper No 2, BSR, 
August 2018). 

Australian Government, Attorney-General’s Department, Review of 
the Privacy Act 1988 - Terms of Reference (30 October 2020) <https:// 
www.ag.gov.au/integrity/publications/review-privacy-act-1988- 
terms-reference>. See also Recommendation 17 - Broader reform 
of Australian privacy law: ACCC, Digital Platforms Inquiry (Final 
Report, June 2019) 476. 

Standards Australia, Final Report: An Artificial Intelligence Standards 
Roadmap: Making Australia’s Voice Heard (Final Report, March 2020); 
Australian Government, Department of Industry, Science, Energy 
and Resources, An Al Action Plan for all Australians: A Call for Views - 
Discussion Paper (2020) 20. 

Jacob Turner, Robot Rules: Regulating Artificial Intelligence (Palgrave 
MacMillan, 2019). 

Jorge Gabriel Jimenez and Margaret Hagan, A Regulatory Sandbox 
for the Industry of Law (March 2019) (Legal Design Lab and Legal 
Executive Institute) 2. 

Centre for Information Policy Leadership, Regulatory Sandboxes in 
Data Protection: Constructive Engagement and Innovative Regulation 
in Practice (8 March 2019). See also CGAP-World Bank, Regulatory 
Sandbox Global Survey: Summary Results (2019). 

See, for example, Katerina Yordanova, ‘The Shifting Sands of 
Regulatory Sandboxes for Al’, KU Leuven Blog (online, 18 July 2019) 
<https://www.law.kuleuven.be/citip/blog/the-shifting-sands-of- 
regulatory-sandboxes-for-ai>. 

ASIC, Enhanced Regulatory Sandbox (Information Sheet 248, August 
2020). 

UK Information Commissioner's Office, Regulatory Sandbox Final 
Report: NHS Digital (December 2020) 3. 

UK Information Commissioner's Office, Regulatory Sandbox Final 
Report: NHS Digital (December 2020) 3. 

UK Information Commissioner's Office, Regulatory Sandbox Final 
Report: NHS Digital (December 2020) 14. 

UK Information Commissioner's Office, Our Key Areas of Focus 

for the Regulatory Sandbox 2020-2021 <https://ico.org.uk/media/ 
for-organisations/documents/26181 12/our-key-areas-of-focus-for- 
regulatory-sandbox.pdf>. 

DP submissions: Microsoft, 7; Standards Australia, 5; Confidential 
submission 42, 15; Australia Institute Centre for Responsible 
Technology, 9; M Kollo, 2; SAS Institute Australia, 14; KPMG 
Australia, 25; University of Melbourne, 5; Capgemini, 9; Australian 
Council of Learned Academies, 4; Australia Industry Group, 9. 

DP submissions: Standards Australia, 5; Microsoft, 7; Capgemini, 9. 
DP submission: Capgemini, 10. 

DP submissions: Standards Australia, 5; R Chalmers, 11. 

DP submission: Capgemini, 10. 

DP submission: University of Melbourne, 5. 

DP submissions: Wiselaw, 9; KPMG Australia, 6; Ethics Matters Pty 
Ltd, 5; SAS Institute Australia, 14-15; Consumer Policy Research 
Centre, 11. 

DP submissions: M Kollo, 2; Confidential submission 42, 15; 

R Chalmers, 11; KPMG Australia, 26. 
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DP submissions: KPMG Australia, 26; Telstra, 1. 

‘CEO Zuckerberg: Facebook's 5 Core Values’, CBS News (online, 

17 May 2012) <https://www.cbsnews.com/news/ceo-zuckerberg- 
facebooks-5-core-values/>. 

Bharath Ganesh, ‘The Silicon Valley Mantra of ‘Move Fast and Break 
Things’ has Fallen Miserably Short’, New Statesman (online, 26 April 
2018) <https://tech.newstatesman.com/guest-opinion/move-fast- 
break-things-mantra>. 

For example, Microsoft's Al chatbot ‘Tay’ started tweeting 
discriminatory and hateful speech within 24 hours of being tested. 
See Elle Hunt, ‘Tay, Microsoft's Al Chatbot, Gets a Crash Course in 
Racism from Twitter,’ The Guardian (online, 24 March 2016) <https:// 
www.theguardian.com/technology/2016/mar/24/tay-microsofts- 
ai-chatbot-gets-a-crash-course-in-racism-from-twitter?CMP=twt_a- 
technology_b-gdntech>. 

The labelling of African Americans as ‘gorillas’ by the Google Photo 
app was identified by users once the app had been launched: Jana 
Kasperkevic, ‘Google Says Sorry for Racist Auto-tag in Photo App’, 
The Guardian (online, 2 July 2015) <https://www.theguardian.com/ 
technology/2015/jul/01/google-sorry-racist-auto-tag-photo-app>. 
OECD, Recommendation of the Council on Artificial Intelligence, OECD/ 
Legal/0449 (adopted 22 May 2019) [2.2]. 

See, for example, the Danish Institute for Human Rights, Human 
Rights Impact Assessment Guidance and Toolbox (2020) 8. 

DP submissions: Element Al, 1; Access Now, 6; Microsoft, 7; 

R Walden, 6; WiseLaw, 7; Ethics Matters Pty Ltd, 7; Maurice 
Blackburn Lawyers, 15; KPMG Australia, 24; Australia Institute's 
Centre for Responsible Technology, 7: D Svantesson; 2; Australian 
Council of Learned Academies, 4; Digital Industry Group Inc, 14. 
DP submissions: |zerobzero, 2; B Auckram, 10. 

DP submissions: Element Al, 6; Ethics Matters Pty Ltd, 4; PwC, 5; 
Australian Council of Learned Academies, 4; KPMG Australia, 21. 
DP Consultation roundtable: 14. 

DP submissions: Australian Academy of Sciences, 6-7; Capgemini, 
6; Castan Centre for Human Rights Law, 25-28; University of 
Technology Sydney, 52. 

DP submissions: Castan Centre for Human Rights Law, 25-28; 

M Kollo, 2; KPMG Australia, 24. DP Consultation roundtable: 14. 
DP submissions: KPMG Australia, 25; R Chalmers, 10; Maurice 
Blackburn Lawyers, 6; Ethics Matters Pty Ltd, 4. 

DP submissions: Ethics Matters Pty Ltd, 4; Castan Centre for 
Human Rights Law, 25-28; R Chalmers, 10; WiseLaw, 8; Microsoft, 5; 
Australian Council of Learned Academies, 4. 

DP submissions: Castan Centre for Human Rights Law 25-28; 
University of Technology Sydney, 53. 

DP submissions: Australian Fair Trade and Investment Network, 1; 
D Svantesson, 2; Telstra, 2. 

DP submissions: R Chalmers, 10; University of Technology 
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643 Australian Government, Office of the Australian Information used to conduct mass surveillance in the West Bank. Microsoft 
Commissioner, Australian Community Attitudes to Privacy Survey 2020 commissioned an independent audit to review these allegations. 
(2020) 84. These allegations were shown to be untrue, and for Microsoft's 
644 DP submissions: Microsoft, 4; KPMG Australia,14; Confidential purposes there could not substantiate a breach of the Microsoft 
submission 42,14. DP Consultation roundtable: 2. Global Finance Portfolio Company Pledge on Facial Recognition. See 
645 R (on the application of Edward Bridges) v The Chief Constable of South Business & Human Rights Resource Centre, Microsoft Criticised for 
Wales Police [2020] EWCA Civ 1058. Investing in Israeli Facial Recognition Company Allegedly Conducting 
646 R (on the application of Edward Bridges) v The Chief Constable of South Surveillance on Palestinians (19 August 2020) <https://www.business- 
Wales Police 12020] EWCA Civ 1058, [164]. humanrights.org/en/microsoft-criticised-for-investing-in-israeli- 
647 R (on the application of Edward Bridges) v The Chief Constable of South facial-recognition-company-allegedly-conducting-surveillance-on- 
Wales Police [2020] EWCA Civ 1058, [91]. palestinians>; M12 Microsoft's Venture Fund, Joint Statement by 
648 R (on the application of Edward Bridges) v The Chief Constable of South Microsoft & AnyVision - AnyVision Audit (27 March 2020) <https://m12. 
Wales Police [2020] EWCA Civ 1058, [199] - [201]. vc/news/joint-statement-by-microsoft-anyvision-anyvision-audit/>. 
649 DP submissions: Access Now, 7; Ethics Matters Pty Ltd, 3; The 664 Arvind Krishna, ‘IBM CEO's Letter to Congress on Racial Justice 
Australia Institute’s Centre for Responsible Technology,9; Al Now Reform’ /BM (8 June 2020) <https://www.ibm.com/blogs/policy/ 
nstitute, 12; Allens Hub for Technology, Law and Innovation, facial-recognition-susset-racial-justice-reforms/>. 
2; Aboriginal and Torres Strait Islander Legal Service (QLD), 3; 665 Amazon, ‘We are Implementing a One Year Moratorium on Police 
WiseLaw, 6; Castan Centre for Human Rights Law, 30; Maurice Use of Rekognition’ (10 June 2020) <https://blog.aboutamazon.com/ 
Blackburn Lawyers, 14. policy/we-are-implementing-a-one-year-moratorium-on-police-use- 
650 DP submissions: Access Now, 8; Castan Centre for Human of-rekognition>. 
Rights Law, 28; Commonwealth Bank of Australia, 7; Herbert 666 See Section 4.5 A framework for regulation. 
Smith Freehills, 2; J Ng, 4; Office of the Australian Information 
Commissioner, 8; SAS Institute Australia, 11. DP Consultation 
roundtables: 2, 4, 6, 10, 16. 
651 DP submission: S Hook, 11. 
652 DP Consultation roundtables: 3, 13. 
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Report of the Special Rapporteur on the Promotion and Protection of 
the Right to Freedom of Opinion and Expression, UN doc A/HRC/41/35 
(28 May 2019); UN Office of the High Commissioner for Human 
Rights, UN Expert Calls For Immediate Moratorium on Sale, Transfer 
and Use of Surveillance Tech (25 June 2019) <https://www.ohchr. 
org/EN/NewsEvents/Pages/DisplayNews.aspx?News|ID=24736>; 

UN Office of the High Commissioner for Human Rights, UN Expert 
Joins Call for Immediate Moratorium on Sale, Transfer and Use 

of Surveillance Tech (15 July 2020) <https://www.ohchr.org/EN/ 
NewsEvents/Pages/DisplayNews.aspx?News|D=26098&LanglD=E>; 
E Tendayi Achiume, Report of the Special Rapporteur on Contemporary 
Forms of Racism, Racial Discrimination, Xenophobia and Related 
Intolerance, UN Doc A/HRC/44/57 (18 June 2020). 

Consultative Committee of the Convention for the Protection 

of Individuals with Regard to Automatic Processing of Personal 
Data, Guidelines on Facial Recognition (Council of Europe, 
T-PD/2020/03rev4, 28 January 2021). 

Consultative Committee of the Convention for the Protection 

of Individuals with Regard to Automatic Processing of Personal 
Data, Guidelines on Facial Recognition (Council of Europe, 
T-PD/2020/03rev4, 28 January 2021) 5. 

For example, the current draft Washington Privacy Act, State 

of Washington, Senate Bill 5528 (2019) proposes a ‘task force 
consisting of fifteen 7 representatives of communities historically 
impacted by surveillance 8 technologies including, but not limited 
to, African American, 9 Hispanic American, Native American, and 
Asian American communities, 10 religious minorities, protest and 
activist groups, and 11 other vulnerable communities, and such 
task force delivers to the 12 legislature a report documenting the 
potential consequences of 13 government use of facial recognition 
systems on the civil rights and civil liberties of vulnerable 
communities’. 

Australian Government, Office of the Australian Information 
Commissioner, What is Personal Information? (5 May 2017) <https:// 
www.oaic.gov.au/privacy/guidance-and-advice/what-is-personal- 
information/>. 

See IP submission: Office of the Australian Information 
Commissioner, 7. OAIC also produces relevant guidance for 
organisations dealing with data, and conducting data analytics; 

see, for example, Australian Government, Office of the Australian 
Information Commissioner, Guide to Data Analytics and the Australian 
Privacy Principles (March 2018). 

Australian Privacy Principle 6.1 provides a right to access ‘personal 
information’ that is ‘about an individual’. The Full Court of the 
Federal Court of Appeal in Privacy Commissioner v Telstra Corporation 
Limited [2017] FCAFC 4 determined that in order for information to 
be considered ‘personal information...about an individual’ within 
the parameters of the Privacy Act, it will need to be determined 

on a case by case basis whether the information requested is an 
about an individual, which ‘will require an evaluative conclusion, 
depending upon the facts of any individual case, just as a 
determination of whether the identity can reasonably be ascertained 
will require an evaluative conclusion’, [63]. 

Victorian Law Reform Commission, Surveillance in Public Places (Final 
Report, 1 June 2010); NSW Law Reform Commission, /nvasion of 
Privacy (Report 120, April 2009); Australian Law Reform Commission, 
Serious Invasions of Privacy in the Digital Era (Report 123, September 
2014). 

Australian Law Reform Commission, Serious Invasions of Privacy in 
the Digital Era (Report 123, September 2014) 51. 

Australian Law Reform Commission, For Your Information: Australian 
Privacy Law and Practice (Report 108, 12 August 2008). 

Australian Law Reform Commission, Serious Invasions of Privacy in 
the Digital Era (Report 123, September 2014) 85. 

Recommendation 19: Statutory tort for serious invasions of privacy: 
ACCC, Digital Platforms Inquiry (Final Report, June 2019) 37. 

ACCC, Digital Platforms Inquiry (Final Report, June 2019) 23. 
Australian Government, Regulation in the Digital Age: Government 
Response and Implementation Roadmap for the Digital Platforms 
Inquiry (2019) 6, 11. 

Australian Government, Regulation in the Digital Age: Government 
Response and Implementation Roadmap for the Digital Platforms 
Inquiry (2019) 6, 11. 
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DP submissions: Information and Privacy Commission NSW, 9; 
Office of the Victorian Information Commissioner, 3; Office of the 
Australian Information Commissioner, 5; Access Now, 7; Digital 
Rights Watch, 2; QUT Digital Media Research Centre, 2; Castan 
Centre for Human Rights Law, 8; Law Council of Australia, 12; J Ng, 
1; University of Melbourne, 4; R Chalmers, 4; WiseLaw, 3; Maurice 
Blackburn Lawyers, 11. DP Consultation roundtable: 16. 

DP submissions: Information and Privacy Commission NSW, 9; 
Access Now, 7; KPMG Australia, 20; Confidential submission 42, 

12; R Chalmers, 4; WiseLaw, 3; J Ng, 1; University of Melbourne, 4; 
Castan Centre for Human Rights Law, 8; Law Council of Australia, 12; 
Australian Law Reform Commission, Serious Invasions of Privacy in 
the Digital Era (Report 123, September 2014); ACCC, Digital Platforms 
Inquiry (Final Report, June 2019). 

DP submissions: Information and Privacy Commission NSW, 9; 
Office of the Victorian Information Commissioner, 3; Maurice 
Blackburn Lawyers, 12. 

DP submission: Medical Insurance Group Australia, 2. 

DP submissions: Amazon Web Services, 2; KPMG Australia, 20; 
Medical Insurance Group Australia, 2; Special Broadcasting Service, 
3. 

ICCPR art 17. 

Noting that the ALRC’s recommendation confined the application 
of the tort to serious invasions of privacy either by intrusion 

upon seclusion or by misuse of private information, the OAIC has 
submitted to the review of the Privacy Act by the Commonwealth 
Attorney-General’s Department that the tort should not be 
confined to the two proposed categories given some serious 
privacy invasions may not fall within them, which ‘will create further 
fragmentation in privacy protections’. See Australian Government, 
Office of the Australian Information Commissioner, Privacy Act 
Review - Issues Paper (Submission by the Office of the Australian 
Information Commissioner) (11 December 2020) 136. 

See, for example, Australian Government, Department of Industry, 
Science, Energy and Resources, An Al Action Plan for all Australians: 
A Call for Views - Discussion Paper (2020). 

See Australian Human Rights Commission, Human Rights 

and Technology (Discussion Paper, December 2019) 68, 69; IP 
submissions: Law Council of Australia; N Suzor, K Weatherall, A 
Daly, AVromen, M Mann; D Glance, M Rizzi; LexisNexis; University 
of Technology Sydney. DP submissions: Element Al, 1; Access Now, 
4; Australian Red Cross, 4; Ethics Matters Pty Ltd, 5; Confidential 
submission 42, 17; R Walden, 7; Aboriginal and Torres Strait Islander 
Legal Service (Qld), 4; Wiselaw; 10; P Gooding; Commonwealth Bank 
of Australia, 9; University of Technology Sydney, 54; NSW Council of 
Civil Liberties, 8; Telecommunications Industry Ombudsman, 1. 

DP submissions: Telstra, 4; Confidential submission 16, 1; SAS 
nstitute Australia, 13; Consumer Policy Research Centre, 12; P 
Gooding; Responsible Technology Australia, 5. DP Consultation 
roundtable: 7. 

DP submissions: Australian Academy of Science; 3; Office of the 
Australian Information Commissioner, 10; Access Now, 5. DP 
Consultation roundtable: 14. 

WP submission: Australian Council of Trade Unions, 4, 5. DP 
submissions: University of Technology Sydney; 54; Aboriginal 

and Torres Strait Islander Legal Service (Qld), 4; Liberty Victoria, 
Australian Privacy Foundation, Queensland Council for Civil 
Liberties, Electronic Frontiers Australia, NSW Council for Civil 
Liberties, 14; KPMG Australia, 28; Commonwealth Bank of Australia, 
9; Wiselaw, 10. 

DP submissions: NSW Bar Association,10; J Thornton, 1. 

WP submission: Australian Council of Trade Unions, 4, 5. DP 
submission: C Curtis, A Radke, J Hereward, 2. 

DP submissions: Access Now, 6; Australian Red Cross, 1; KPMG 
Australia, 3; Community and Public Sector Union, 1; Microsoft, 8; 
SafeGround Inc; The Australia Institute's Centre for Responsible 
Technology, 3; Aboriginal and Torres Strait Islander Legal Service 
(Qld), 2; Public Interest Advocacy Centre, 3; Telstra, 4; Law Council 
of Australia, 6; Castan Centre for Human Rights Law, 8; S Hook, 55; 
University of Technology Sydney, 2; Maurice Blackburn Lawyers, 1; 
C Curtis, A Radke, J Hereward, 4; J Crock, 14; Telecommunications 
Industry Ombudsman, 1. 

WP submission: Office of the eSafety Commissioner, 3. 
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See WP submissions: Consumer Policy Research Centre, 3; Effective 
Altruism ANZ, 12. 

WP Symposium, 6 March 2019. 

See, for example, WP submissions: J Powles, M Rizzi, F McGaughey, 
D Glance, 1, 2; G Greenleaf, R Clarke, D Lindsay, 1; Allens Hub for 
Technology, Law and Innovation, 7; WP Symposium, 6 March 2019. 
DP submissions: KPMG Australia, 29; University of Technology 
Sydney, 55; WP Symposium, 6 March 2019. 

See, for example, IP submissions: Castan Centre for Human Rights 
Law, 2; University of Melbourne, 2. See WP submissions: Microsoft; 
Digital Industry Group Inc, 4. DP submissions: Australian Industry 
Group, 1; Digital Industry Group Inc, 13. 

WP submission: Allens Hub for Technology, Law, and Innovation, 3. 
DP submissions: Microsoft, 8; Digital Industry Group Inc, 13. 

See, for example, Australian Government, Innovation and Science 
Australia, Australia 2030: Prosperity Through Innovation (2017). 

See Chapters 2, 3, 5. 

DP submission: Office of the Australian Information Commissioner, 
10. DP Consultation roundtable 1. WP submission: Office of the 
Victorian Information Commissioner, 2. 

DP submission: Office of the Australian Information Commissioner, 
10. 

DP submission: Consumer Policy Research Centre, 22. 

DP submission: Element Al, 4. 

DP submissions: Office of the Australian Information Commissioner, 
10; Access Now, 4; Australian Red Cross, 4; Aboriginal and Torres 
Strait Islander Legal Service (Qld) 4; University of Technology 
Sydney, 54. 

DP submissions: Commonwealth Bank of Australia, 9; 

R Chalmers,13; Access Now, 5. 

DP submission: Digital Industry Group Inc, 13. 

See, for example, Australian Human Rights Commission and World 
Economic Forum, Artificial Intelligence: Governance and Leadership 
(White Paper, January 2019); Council of Europe Commissioner for 
Human Rights, Unboxing Artificial Intelligence: 10 Steps to Protect 
Human Rights (May 2019). 

Toby Walsh, Neil Levy, Genevieve Bell, Anthony Elliott, James 
Maclaurin, Iven Mareels and Fiona Wood, The Effective and Ethical 
Development of Artificial Intelligence: An Opportunity to Improve our 
Wellbeing (Report for the Australian Council of Learned Academies, 
July 2019) 13. 

DP submission: Office of the Australian Information Commissioner, 
10. WP submissions: Centre for Policy Futures, 6; H Dobson, 5; 

J Ford, 16; G Greenleaf, R Clarke, D Lindsay, 5. 

DP submissions: D Svantesson, 2; University of Melbourne; 6. 

WP submission: Remi Al, 5. 

See, for example, Anna Brown, Alexandra Chouldechova, Emily 
Putnam-Hornstein and Andrew Tobin, ‘Toward Algorithmic 
Accountability in Public Services: A Qualitative Study of Affected 
Community Perspectives on Algorithmic Decision-making in Child 
Welfare Services’, (Conference Paper, Conference on Human Factors 
in Computing Systems, May 2019) 1; Maciej Kuziemski and Gianluca 
Misuraca, ‘Al Governance in the Public Sector: Three Tales From the 
Frontiers of Automated Decision-making in Democratic Settings’ 
(2020) 44(6) Telecommunications Policy 101976. 

DP submissions: Office of the Victorian Information Commissioner, 
2; Liberty Victoria, Australian Privacy Foundation, Queensland 
Council for Civil Liberties, Electronic Frontiers Australia, NSW Council 
for Civil Liberties, 14; University of Melbourne, 6. 

DP submissions: University of Melbourne, 6; Australian Industry 
Group, 3; Consumer Policy Research Centre, 9. 

DP submission: KPMG Australia, 28. 

UK Centre for Data Ethics and Innovation, About Us <http://www. 
gov.uk/government/organisations/centre-for-data-ethics-and- 
innovation/about>. 

See IP submissions: L Fowkes, 7; M Paterson; M Wildenauer; Allens 
Hub for Technology, Law and Innovation; Choice, 2. DP submissions: 
Access Now, 4; Element Al, 5; T Aulich, 2. 

DP submission: Aboriginal and Torres Strait Islander Legal Service 
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725 


726 


727 
728 
729 


730 
731 


732 


733 
734 
735 


736 


737 
738 


739 
740 
741 
742 
743 
744 
745 
746 
747 
748 


749 


750 


751 


752 


753 
754 


755 


756 
757 


758 


759 


760 


Stefano Baruffaldi, Brigitte van Beuzekom, Helene Dernis and 
Dietmar Harhoff, /dentifying and Measuring Developments in 
Artificial Intelligence: Making the Impossible Possible (OECD Science, 
Technology and Industry Working Paper, May 2020) 8. 

DP submissions: NSW Bar Association, 10; Digital Rights Watch, 2; 
NSW Council of Civil Liberties, 8. DP Consultation roundtable: 1. 
DP submissions: NSW Bar Association, 10; Digital Rights Watch, 2. 
DP submission: NSW Council of Civil Liberties, 8. 

UK Information Commissioner's Office, Guidance on the Al Auditing 
Framework: Draft Guidance for Consultation (2020) <https://ico.org. 
uk/media/2617219/guidance-on-the-ai-auditing-framework-draft- 
for-consultation.pdf>; see also, ASIC, ASIC's Compulsory Information 
Gathering Powers (Information Sheet 145, March 2020); Amnesty 
International and Access Now, The Toronto Declaration (16 May 
2018). 

DP submission: Digital Industry Group Inc, 7. 

See Exposure Draft: Data Availability and Transparency Bill 2020, 
Part 4.3 <https://www.datacommissioner.gov.au/sites/default/ 
files/2020-09/Data%20Availability%20and%20Transparency%20 
Bill%202020_exposure%20draft.pdf>. 

See Australian Human Rights Commission, Human Rights and 
Technology (Discussion Paper, December 2019) 131. 

See Chapter 5. 

See Chapters 6 and 7. 

Australian Human Rights Commission, Using Artificial Intelligence to 
Make Decisions: Addressing the Problem of Algorithmic Bias (Technical 
Paper, November 2020). 

DP submissions: Al Now Institute, 10; Access Now, 5; Actuaries 
Institute, 3; Microsoft, 9. 

DP submission: Confidential submission 16, 1. 

DP submissions: Medical Insurance Group Australia, 4; KPMG 
Australia, 11. 

DP submission: KPMG Australia, 23. 

DP submission: SAS Institute Australia, 13. 

See Recommendation 4. 

See Recommendation 7. 

See Recommendation 9. 

See Recommendation 14. 

See Recommendation 15. 

See Recommendation 16. 

See Recommendation 17. 

See Australian Government, eSafety Commissioner, Our Legislative 
Functions <https://www.esafety.gov.au/about-us/who-we-are/our- 
legislative-functions>. 

Enhancing Online Safety Act 2015 (Cth) s 67. 

Australian Human Rights Commission Act 1986 (Cth) pt 22AA National 
Children’s Commissioner. 

See Australian Human Rights Commission, About <https:// 
humanrights.gov.au/about>. 

DP submissions: University of Technology, 54; Maurice Blackburn 
Lawyers, 16; Commonwealth Public Sector Union, 38; Liberty 
Victoria, Australian Privacy Foundation, Queensland Council for 
Civil Liberties, Electronic Frontiers Australia, NSW Council for Civil 
Liberties, 14. DP Consultation roundtable: 6. 

DP submission: Responsible Technology Australia, 5. 

DP submissions: Australian Academy of Science; 3; Digital Industry 
Group Inc, 7; Office of the Australian Information Commissioner, 10. 
DP submission: Australian Academy of Sciences, 3. 

DP submission: Law Council of Australia, 6. 

DP submissions: NSW Bar Association,10; Element Al, 6; Australian 
Red Cross, 4; KPMG Australia, 28; P Gooding; WiseLaw, 10; Access 
Now, 4; Aboriginal and Torres Strait Islander Legal Service (Qld), 4; 
Confidential submission 42, 17; R Walden, 7; Telstra, 4; University of 
Technology Sydney, 54; Access Now, 4. 

DP submissions: Australian Academy of Science, 3; Consumer Policy 
Research Centre, 9; Confidential submission 16, 1. 

DP submissions: R Chalmers,13; Commonwealth Bank of Australia, 
9. 

DP submission: Office of the Australian Information 
Commissioner,10. 
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Stakeholders broadly referred to these two types of access 
throughout the consultation process. See, for example, IP 
submission: Digital Gap Initiative, 8. DP submission: Australian 
Communications Consumer Action Network, 18. 

CRPD art 1. All people, including people with disability, have the right 
to equality and non-discrimination: /CCPR arts 2, 26; ICESCR art 2. 
CRPD art 9 is informed by the broad definition of ‘communication’ 
in art 2 and the principles in art 3. For the purposes of art 9, 

ICT includes any information and communication device or 
application and its content: Committee on the Rights of Persons 
with Disabilities, General Comment No 2 (2014) Article 9: Accessibility, 
11% sess, CRPD/C/GC/2 (11 April 2014) 2. The CRPD sets out durable 
principles that apply as technology develops. 

This CRPD-protected right is also related to the right to benefit from 
scientific progress: /CCPR art 15(1)(b). 

CRPD arts 9(2)(a), 9(h). The term ‘universal design’ is defined in CRPD 
art 2. 

Excerpts have been adapted from: Disability Action, What is the 
UNCRPD? <https://www.disabilityaction.org/what-is-the-uncrpd>. 
DP submissions: Digital Gap Initiative; Australian Communications 
Consumer Action Network. DP Consultation roundtables: 17, 18, 
19, 20, 21. Some stakeholders expressed endorsement for the 
framework, as well as a statement of broad support for all or many 
of the Discussion Paper Part D proposals. DP submissions: KPMG 
Australia, 30; Ethics Matters Pty Ltd, 5; Liberty Victoria, Australian 
Privacy Foundation, Queensland Council for Civil Liberties, 
Electronic Frontiers Australia, NSW Council for Civil Liberties, 

14; Consumer Policy Research Centre; WiseLaw, 10-13; Maurice 
Blackburn Lawyers, 1; Digital Rights Watch, 6; Commonwealth Bank 
of Australia, 9. See also IP submissions: Australian Rehabilitation 
and Assistive Technology Association, 3; Northraine, 13; Digital 

Gap Initiative, 4; Intopia, 10; Ability Research Centre, 2; National 
Disability Services, 1; University of Technology Sydney, 65; Carers 
Australia, 1; Public Interest Advocacy Centre, 16. 

IP submission: Australian Communications Consumer Action 
Network, 6. 

CRPD art 3. 

Committee on the Rights of Persons with Disabilities, General 
Comment No 2 (2014) Article 9: Accessibility, 11" sess, CRPD/C/GC/2 
(11 April 2014) 2. This Committee noted that the general right to 
access places and services is reflected in the international human 
rights framework. 

UNESCO, Education for People and Planet: Creating Sustainable Futures 
for All (Global Education Monitoring Report, 2016) 8. 

See, for example, DP submissions: Digital Gap Initiative; Blind 
Citizens Australia; Public Interest Advocacy Centre; R Walden; 
Consumer Policy Research Centre. IP submissions: Australian 
Communications Consumer Action Network; Kingsford Legal Centre; 
National Association of Community Legal Centres. 

Written submissions and consultation feedback from IP and DP 
public consultation phases. 

Bias and discrimination in automated decision making are 
considered in detail at Chapter 8. 

DP submissions: Digital Gap Initiative, 10-11; NSW Bar Association 
12-16. DP Consultation roundtables: 17, 18, 19, 20, 21. 

IP submission: National Association of Community Legal Centres, 
10. 

DP Consultation roundtables: 17, 18, 19, 20, 21. 

DP submission: NSW Bar Association, 12-16. 

DP Consultation roundtables: 17, 21. 

DP Consultation roundtable: 17. 

DP Consultation roundtable: 17. 

CRPD arts 4, 31, 33. 

IP submission: S Murphy, 6. 

DP submission: Blind Citizens Australia, 5. IP submission: S Murphy, 
6. 

IP submission: Confidential submission 38, 2. 

IP submission: ANZ, 1. 

DP submission: Blind Citizens Australia, 5. IP submission: ANZ, 1. 
Written submissions and consultation feedback from IP and DP 
public consultation phases. 
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IP submissions: National Association of Community Legal Centres, 4, 
10; Intellectual Disability Rights Service, 3-4; Public Interest Advocacy 
Centre, 16; Speech Pathology Australia, 4. 

IP submissions: Public Interest Advocacy Centre, 16; S Murphy, 7. 
IP submission: S Murphy, 7. 

IP submission: Public Interest Advocacy Centre, 16. 

IP submission: Intellectual Disability Rights Service, 3-4. 

IP submission: Speech Pathology Australia, 4. 

See, for example, DP submissions: Digital Gap Initiative; Australian 
Communications Consumer Action Network; Blind Citizens 
Australia; A Normand; Public Interest Advocacy Centre; Confidential 
submission 42, S Hook. DP Consultation roundtables: 17, 18, 19, 20, 
21. 

See DP submissions: Blind Citizens Australia, 2; Australian 
Communications Consumer Action Network, 22; Digital Gap 
Initiative, 24; Public Interest Advocacy Centre, 5. DP Consultation 
roundtables: 17, 18, 19, 20, 21. 

See, for example, DP submission: Digital Gap Initiative, 8-9. DP 
Consultation roundtables: 18, 20, 21. 

Disability Discrimination Act 1992 (Cth) s 24. 

Australian Human Rights Commission, Complaints under the 
Disability Discrimination Act (14 December 2012) <https://www. 
humanrights.gov.au/complaints/complaint-guides/complaints- 
under-disability-discrimination-act>. 

See, for example, Anti-Discrimination Act 1977 (NSW), Human Rights 
Act 2004 (ACT). 

As noted in DP submission: Digital Gap Initiative, 12-13. 

DP consultation roundtable: 19. IP submissions: Australian 
Communications Consumer Action Network, 11; Public Interest 
Advocacy Centre, 16. 

P submission: Australian Communications Consumer Action 
Network, 11. 

P submission: P Harpur, 41. 

P submission: University of Technology Sydney, 69. 

P submission: Digital Gap Initiative, 5. 

DP submissions: Blind Citizens Australia, 2; Australian 
Communications Consumer Action Network, 22; Digital Gap 
nitiative, 24; Public Interest Advocacy Centre, 5. DP Consultation 
roundtables: 17, 18, 19, 20, 21. One industry representative group, 
Digital Industry Group Inc, was broadly supportive of the inclusion 
of accessible Digital Communication Technology under the DDA, but 
suggested any new standard draw upon existing standards. 
Disability Discrimination Act 1992 (Cth) s 31. 

P submission: Kingsford Legal Centre, 15. 

Standards Australia, About Us <https://www.standards.org.au/>. 

P submission: Digital Gap Initiative, 13. 

DP submission: Digital Gap Initiative, 24. 

DP submission: Blind Citizens Australia, 2. 

DP submissions: Australian Communications Consumer Action 
Network; Blind Citizens Australia; A Normand; Digital Gap Initiative; 
S Hook; IP submissions: Public Interest Advocacy Centre; P Harpur; 
A Normand; Digital Gap Initiative; S Murphy. 

See, for example, DP submissions: SAS Institute Australia, 18; 

A Normand, 6. A Normand, for example, stated the introduction of 
a flat screen interface for a mobile ‘was at first baffling for many 
users and certainly didn’t fall withing the boundaries of existing 
standards. Today, the use of gestures on mobile phones has been 
widely adopted by screen reader users, resulting in vastly improved 
access to technology’. 

See, for example, DP submissions: Public Interest Advocacy 
Centre, 5; WiseLaw, 71; Australian Communications Consumer 
Action Network, 22; Digital Gap Initiative, 12; Carers NSW, 3. DP 
Consultation roundtable: 20. 

IP submission: Digital Gap Initiative, 17. 

IP submission: LexisNexis, 23-24. 

IP submission: LexisNexis, 23-24. 

Senate Community Affairs References Committee, Commonwealth 
of Australia, Delivery of Outcomes Under the National Disability 
Strategy 2010-2020 to Build Inclusive and Accessible Communities 
(2017). 

Australian Government, Shut Out: The Experience of People with 
Disabilities and their Families in Australia (2009). 

CRPD art A(a). 


Human Rights and Technology Final Report * 2021 ° 233 


234 


Endnotes 


830 


831 


832 


833 


834 


835 


836 


837 
838 
839 
840 


841 


842 


843 


844 


845 


CRPD art 33. 

Committee on the Rights of Persons with Disabilities, Concluding 
Observations on the Combined Second and Third Periodic Reports of 
Australia, CRPD/C/AUS/CO/2-3 (15 October 2019) 5. 

Australian Human Rights Commission, Disability Standards 

(1 November 2016) <https://www.humanrights.gov.au/our-work/ 
disability-rights/disability-standards>. 

See, for example, DP submissions: Digital Gap Initiative, 25; 
Australian Communications Consumer Action Network, 22. 

See, for example, Disability Standards for Accessible Public Transport 
2002 (Cth) pt 34: Review, Schedule 1: Target dates for compliance. 
See Section 15.2 on Human Rights by design. 

Committee on the Rights of Persons with Disabilities, General 
Comment No 2 (2014) Article 9: Accessibility, 11 sess, CRPD/C/GC/2 
(11 April 2014) 5. 

The Commission has previously recorded similar concerns. See 
Australian Human Rights Commission, Submission No 30 to the 
Department of Infrastructure, Regional Development and Cities, 
Review of the Disability Standards for Accessible Public Transport 2002 
(December 2018) 15; and Australian Human Rights Commission, 
Submission to the UN Committee on the Rights of Persons with 
Disabilities, Information Concerning Australia’s Compliance with the 
Convention on the Rights of Persons with Disabilities (25 July 2019), in 
which it noted the lack of measures to ensure nationally consistent 
implementation, enforceability, monitoring and compliance under 
the Building Standards and Transport Standards. 

Committee on the Rights of Persons with Disabilities, Concluding 
Observations on the Combined Second and Third Periodic Reports of 
Australia, CRPD/C/AUS/CO/2-3 (15 October 2019) 5. 

See, for example, Australian Government, Department of Education, 
Employment and Workplace Relations, Report on the Review of the 
Disability Standards for Education 2005 (2012). 

See, for example, Australian Human Rights Commission, Submission 
to the UN Committee on the Rights of Persons with Disabilities, 
Information Concerning Australia’s Compliance with the Convention on 
the Rights of Persons with Disabilities (25 July 2019). 

Committee on the Rights of Persons with Disabilities, Concluding 
Observations on the Combined Second and Third Periodic Reports of 
Australia, CRPD/C/AUS/CO/2-3 (15 October 2019) 5. 

The Commission has previously recorded similar concerns relating 
to the DDA—for example, in the Disability Standards Public 
Transport Review. 

See, for example, IP submission: Digital Gap Initiative, 11, which 
noted the positive steps taken by the NSW Government and the 
Digital Transformation Agency regarding accessible services and 
facilities. 

See, for example, IP submissions: Intellectual Disability Rights 
Service, 3-4; Speech Pathology Australia, 4. 

See IP submissions: Digital Gap Initiative, 11; Intopia, 6; WebKeyIT, 6. 
IP submission: Intellectual Disability Rights Service, 3-4. 

IP submission: Speech Pathology Australia, 7. 

Digital Transformation Agency, Make it Accessible <https://www. 
dta.gov.au/help-and-advice/digital-service-standard/digital-service- 
standard-criteria/9-make-it-accessibleAccessibility ICT Procurement 
Standard>. 

Standards Australia, Standards Catalogue <https://www.standards. 
org.au/standards-catalogue/sa-snz/communication/it-040/as--en-- 
301-dot-549-colon-2016>. 

Digital Transformation Agency, Make it Accessible <https://www. 
dta.gov.au/help-and-advice/digital-service-standard/digital-service- 
standard-criteria/9-make-it-accessibleAccessibility ICT Procurement 
Standard>. 

W3C Web Accessibility Initiative, Web Content Accessibility Guidelines 
(WCAG) Overview (15 June 2018) <https://www.w3.org/WAI/ 
standards-guidelines/wcag/>. 

Department of Finance and Deregulation, Web Accessibility National 
Transition Strategy (2010). 

For a 2012 Progress Report, see Australian Government, 
Department of Finance, Web Accessibility National Transition Strategy 
2012 Progress Report Data (18 February 2017) <https://data.gov.au/ 
dataset/ds-dga-e59c06a1-86e6-4ac7-8c34-d52f4d2406d8/details>. 
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Vivienne Conway, ‘Website Accessibility in Australia and the National 
Transition Strategy: Outcomes and Findings’ (Doctoral thesis, Edith 
Cowan University, 2014) iv. 

DP submissions: Digital Gap Initiative, 19; Australian 
Communications Consumer Action Network, 18; Blind Citizens 
Australia, 2; SAS Institute Australia, 28; Public Interest Advocacy 
Centre, 2; Microsoft, 9; Carers NSW, 3. DP Consultation roundtables: 
17, 18, 19, 20, 21. 

DP submissions: Public Interest Advocacy Centre, 2; Blind Citizens 
Australia, 2; Digital Gap Initiative, 19; Australian Communications 
Consumer Action Network, 18. DP Consultation roundtables: 17, 18, 
19, 20, 21. 

DP submission: Digital Gap Initiative, 19. 

DP submission: Digital Gap Initiative, 19. IP submission: S Murphy, 9. 
DP Consultation roundtables: 19, 21. 

DP submissions: Digital Gap Initiative, 19; A Normand, 3. 

DP submission: A Normand, 3. 

IP submission: ANZ, 2. 

IP submissions: Portable, 3; Australian Rehabilitation and Assistive 
Technology Association, 10. 

See, for example, Government of South Australia, Online Accessibility 
Toolkit <https://www.accessibility.sa.gov.au>. 

CRPD art 9. 

Committee on the Rights of Persons with Disabilities, General 
Comment No 2 (2014) Article 9: Accessibility, 11" sess, CRPD/C/GC/2 
(11 April 2014) 9-10. 

Committee on the Rights of Persons with Disabilities, Concluding 
Observations on the Combined Second and Third Periodic Reports of 
Australia, CRPD/C/AUS/CO/2-3 (15 October 2019) 5. 

World Economic Forum, Guidelines for Al Procurement (White Paper, 
September 2019) <http://www3.weforum.org/docs/WEF_Guidelines_ 
for_Al_Procurement.pdf> 4; Gerard Goggin, Scott Hollier and Wayne 
Hawkins, ‘Internet Accessibility and Disability Policy Lessons for 
Digital Inclusion and Equality from Australia’ (2017) 6(1) Internet 
Policy Review 1, 9. 

Australian Government, Commonwealth Indigenous Procurement 
Policy (2015) 19. 

Commonwealth, Department of the Prime Minister and Cabinet, 
Third Year Evaluation of the Indigenous Procurement Policy (2019) 3. 
Detailed data for 2019-20 Commonwealth Indigenous 

procurement outcomes is available at: Australian Government, 
National Indigenous Australians Agency, /ndigenous Procurement 
Policy <https://www.niaa.gov.au/indigenous-affairs/economic- 
development/indigenous-procurement-policy-ipp#latest-news>. 
Business Council of Australia, Indigenous Engagement <https://www. 
bca.com.au/indigenous_engagement>. 

See Australian Government, Department of Finance, Commonwealth 
Procurement Rules (20 April 2019). 

See, for example, NSW Government, Procurement Policy Framework 
(October 2020). 

For more information on people with disability and employment 
see Australian Human Rights Commission, Willing to Work: National 
Inquiry into Employment Discrimination Against Older Australians and 
Australians with Disability (2016). 

Australian Bureau of Statistics, Disability, Ageing and Carers, Australia: 
Summary of Findings (24 October 2019) <https://www.abs.gov.au/ 
statistics/health/disability/disability-ageing-and-carers-australia- 
summary-findings/latest-release>. 

Some of these principles are considered in World Economic Forum, 
Guidelines for Al Procurement (White Paper, September 2019) <http:// 
www3.weforum.org/docs/WEF_Guidelines_for_Al_Procurement. 
pdf>. 

DP submissions: Digital Gap Initiative, 20; Australian 
Communications Consumer Action Network, 18; SAS Institute 
Australia, 18; Microsoft, 9. 

IP submission: Intopia, 6. 

DP submission: Vision Australia. IP submissions: S Murphy, 10; 
University of Technology Sydney, 72; Digital Gap Initiative, 6. 

See IP submissions: S Murphy, 6; University of Technology Sydney, 
72. 

IP submission: Australian Communications Consumer Action 
Network, 5. 
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See IP submissions: S Murphy, 6; University of Technology Sydney, 
72. DP Consultation roundtable: 14. 

DP submission: Telstra, 5. 

As discussed in Chapter 15. 

Yvette Maker, Alex Callahan, Bernadette McSherry, Jeannie Marie 
Paterson, Lisa Brophy and Anna Arstein-Kerslake, /mproving Access 
and Support for Consumers with Cognitive Disabilities (University of 
Melbourne, 2018). 

Australian Government, Department of Industry, Science, Energy 
and Resources, Corporate Plan 2019-20 (2019) iv. 

Digital Transformation Agency, About Us <https://www.dta.gov.au/ 
about-us>. 

Digital Transformation Agency, Digital Service Standard Criteria 
<https://www.dta.gov.au/help-and-advice/digital-service-standard/ 
digital-service-standard-criteria>. 

CRPD arts 9, 30(1)(b). 

Australian Human Rights Commission, Human Rights and Technology 
(Discussion Paper, December 2019) Proposal 22. 

DP submissions: Australian Communications Consumer Action 
Network, 19; Blind Citizens Australia, 4; SAS Institute Australia, 26; 
QUT Digital Media Research Centre, 4. Digital Gap Initiative, 20-21, 
broadly supported this position, but recommended a starting 
minimum quota of 10% of content. DP Consultation roundtables: 
17, 18, 19, 20, 2. 

See, for example, Audio Description in Australia <http:// 
audiodescriptionau.com.au/>; Blind Citizens Australia, Tv4all <http:// 
www.tv4all.com.au/>. 

Vision Australia, Audio Description Finally Arrives on Australian Free to 
Air TV (24 June 2020) <https://www.visionaustralia.org/community/ 
news/2020-07-01/audio-description-finally-arrives-australian-free- 
air-tv?mc_cid=129e36cd0b&mc_eid=95799fd1b6>. 

Paul Fletcher MP, ‘Audio Description on ABC and SBS Delivers 
Greater Accessibility’ (Media Release, 22 June 2020). 

Australian Government, Department of Communications and the 
Arts, Audio Description Working Group—Final Report (2017) 10. 

DP submission: Australian Communications Consumer Action 
Network, 19. DP Consultation roundtables: 17, 18, 19, 20, 21. 

DP Consultation roundtable: 18. 

DP Consultation roundtable: 18. 

DP Consultation roundtables: 17, 18, 19, 20, 21. 

DP submission: Blind Citizens Australia, 4. DP Consultation 
roundtables: 17, 18, 19, 20, 21. 

DP submissions: Australian Communications Consumer Action 
Network, 19; Blind Citizens Australia, 4. 

DP Consultation roundtable: 18. 

DP submission: Australian Communications Consumer Action 
Network, 19. DP Consultation roundtables: 17, 20. 

DP submissions: Australian Communications Consumer Action 
Network, 19; Digital Gap Initiative, 20. DP Consultation roundtables: 
17, 18, 19, 21. Other stakeholders also supported an increase in 
captioning quotas; DP submissions: Public Interest Advocacy Centre, 
2; SAS Institute Australia, 26; QUT Digital Media Research Centre, 4. 
‘Free-to-air television broadcasters’ includes commercial television 
broadcasting licensees and national television broadcasters (ABC, 
SBS). Broadcasting Services Act 1992 (Cth) s 130ZR. 

Some broadcasters have a primary broadcast channel and other 
multi-channels. For example, Channel 7 is the primary broadcast 
channel, and 7mate, 7TWO and 7flix are multi-channels. 

DP submission: Australian Communications Consumer Action 
Network, 19. 

DP Consultation roundtables: 18, 20, 21. 

Australian Government, Department of Communications and the 
Arts, Audio Description Working Group—Final Report (2017); Katie 
Ellis, Gwyneth Peaty, Leanne McRae, Mike Kent and Kathryn Locke, 
Audio Description in Australia (Australian Communications Consumer 
Action Network, 2019). 

CRPD art 4(3). 

DP submission: Australian Communications Consumer Action 
Network, 19. 

See ABC, Audio Description Schedule <https://www.abc.net.au/tv/ 
audiodescription/>; SBS, Audio Description Services <https://www.sbs. 
com.au/aboutus/audio-description-services>. 
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Katie Ellis, Gwyneth Peaty, Leanne McRae, Mike Kent and Kathryn 
Locke, Audio Description in Australia (Australian Communications 
Consumer Action Network, 2019) 42. 

Katie Ellis, Gwyneth Peaty, Leanne McRae, Mike Kent and Kathryn 
Locke, Audio Description in Australia (Australian Communications 
Consumer Action Network, 2019) 13, 63-65; Australian Government, 
Department of Communications and the Arts, Audio Description 
Working Group—Final Report (2017) 14. 

Katie Ellis, Gwyneth Peaty, Leanne McRae, Mike Kent and Kathryn 
Locke, Audio Description in Australia (Australian Communications 
Consumer Action Network, 2019) 21. 

Australian Government, Department of Communications and the 
Arts, Audio Description Working Group—Final Report (2017) 11 
Australian Government, Department of Communisations and the 
Arts, Audio Description Working Group—Final Report (2017) 11. 

See, for example, Audio Description in Australia <http:// 
audiodescriptionau.com.au/>; Blind Citizens Australia, Tv4all <http:// 
www.tv4all.com.au/>. 

Broadcasting Services Amendment (Audio Description) Bill 2019 
(Cth) Item 14. 

Australian Government, Department of Communications and the 
Arts, Audio Description Working Group—Final Report (2017) 25. 
Australian Government, Department of Communications and the 
Arts, Audio Description Working Group—Final Report (2017) 39. 

SBS provided feedback to the Commission on the consultation 

and implementation process. See also: ABC, ABC Launches Audio 
Description Service for Blind or Vision-impaired Audiences <https:// 
about.abc.net.au/press-releases/abc-launches-audio-description- 
service-for-blind-or-vision-impaired-audiences/>. 

National (ABC and SBS) and commercial broadcasters make up free- 
to-air TV in Australia, and subscription broadcasting is an extra paid 
service (eg Foxtel). 

ACMA, Report on the Review of the Captioning Obligations in the 
Broadcasting Services Act 1992 (2017) 7-8. 

ACMA, Report on the Review of the Captioning Obligations in the 
Broadcasting Services Act 1992 (2017) 7-8. 

ACMA, Report on the Review of the Captioning Obligations in the 
Broadcasting Services Act 1992 (2017) 7-8. 

See Broadcasting Services Act 1992 (Cth) s 130ZV; ACMA, Provide 
Captions on Subscription TV <https://www.acma.gov.au/provide- 
captions-subscription-tv>. 

See ACMA, Provide Captions on Subscription TV <https://www.acma. 
gov.au/provide-captions-subscription-tv>. 

See freeview, TV Guide <https://www.freeview.com.au/tv-guide>. 
For categories of subscription channels see: ACMA, Provide Captions 
on Subscription TV <https://www.acma.gov.au/provide-captions- 
subscription-tv>. 

DP Consultation roundtables: 17, 18, 19. 

See, for example, a recommendation to make captioning on SVOD 
compulsory under federal legislation if providers did not voluntarily 
provide acceptable levels of captioning by the end of 2016: Media 
Access Australia, Access on Demand: Captioning and Audio Description 
on Video on Demand Services (Report Series, April 2015). 

See, for example, Vision Australia, Audio Described Movies on 
Streaming Services (7 May 2020) <https://www.visionaustralia.org/ 
community/news/2020-05-07/audio-described-movies-streaming- 
services>; American Council of the Blind, Audio Description Project 
<https://www.acb.org/adp/>. 

See, for example, Vision Australia, At the Cinema <https://www. 
visionaustralia.org/community/events/audio-description-services/ 
at-the-cinema>; Open Captions Australia, Open Captions <https:// 
opencaptions.com.au/open-captions/>; Event Cinemas, Accessibility 
<https://www.eventcinemas.com.au/promotions/accessibility>. 
Vision Australian, At the Cinema <https://www.visionaustralia.org/ 
community/events/audio-description-services/at-the-cinema>. 
Commonwealth, Media Reform Green Paper: Modernising Television 
Regulation in Australia (2020). 

These concerns are outlined in Letter from Deaf Australia to Prime 
Minister Scott Morrison, 12 March 2020 <https://www.afdo.org.au/ 
deaf-australia-letter/>. 

Commonwealth, Report on the Review of the Captioning Obligations in 
the Broadcasting Services Act 1992 (2017) 18. 
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See Australian Subscription Television and Radio Association, 
Broadcast of Emergency Information <https://www.astra.org.au/ 
advocacy/broadcast-of-emergency-information>; Free TV Australia, 
Advisory Note Broadcast of Emergency Information <https://www. 
freetv.com.au/wp-content/uploads/2019/12/FINAL-Amended- 
Advisory-Note-Emergency-Information-12-June.pdf>. 

The Australian Subscription Television and Radio Association 
(ASTRA) and Free TV Australia made an undertaking to Victorian 
Deaf Society in relation to ensuring an Auslan interpreter is included 
in the broadcast if they are present at the announcement. See 
Commonwealth, ACMA, Report on the Review of the Captioning 
Obligations in the Broadcasting Services Act 1992 (2017) 19. 

CRPD art 11. 

Letter from Deaf Australia to Prime Minister Scott Morrison, 

12 March 2020 <https://www.afdo.org.au/deaf-australia-letter/>. 
NSW Department of Premier and Cabinet, Final Report of the NSW 
Bushfire Inquiry (2020) 353. See also Royal Commission into National 
Natural Disaster Arrangements (Final Report, 28 October 2020) [13.7]. 
Royal Commission into National Natural Disaster Arrangements (Final 
Report, 28 October 2020) [13.71]. 

Commonwealth, Department of Health, Management and 
Operational Plan for People with Disability (2020) 28. 

DP submission: Digital Gap Initiative, 12. DP Consultation 
roundtables 18, 19, 21. 

Katie Ellis, Kai-Ti Kao, Gwyneth Peaty and Kathryn Locke, Live 
Caption Quality Monitoring on Australian Free-to-Air Television (ACMA, 
November 2019). 

See recommendations 1 and 2. Katie Ellis, Kai-Ti Kao, Gwyneth Peaty 
and Kathryn Locke, Live Caption Quality Monitoring on Australian Free- 
to-Air Television (ACMA, November 2019). 

ACMA, Captions That TV Broadcasters Must Provide (2 September 
2020) <https://www.acma.gov.au/captions-tv-broadcasters-must- 
provide>; Broadcasting Services (Television Captioning) Standard 2013. 
Broadcasting Services (Television Captioning) Standard 2013 s 6. 
ACMA, Broadcaster Compliance with TV Captioning Obligations (13 
August 2020) <https://www.acma.gov.au/broadcaster-compliance- 
tv-captioning-obligations>. 

ACMA, Complain About Captioning on TV (9 December 2019) <https:// 
www.acma.gov.au/complain-about-captioning-tv>. 

CRPD arts 31, 33. 

See IP submissions: La Trobe LawTech, 9; National Association of 
Community Legal Centres, 4; Australian Red Cross, 8. 

IP submission: Law Council of Australia, 6. 

The Australian Digital Inclusion Index measures progress in 
addressing barriers to digital inclusion, including those relating to 
access, affordability and digital ability. ‘Digital ability’ is a concept 
that refers to a person's attitudes, skills and activities in the digital 
realm: Australian Digital Inclusion Index, Measuring Australia’s Digital 
Divide: The Australian Digital Inclusion Index 2020 (2020) 10. 

See, for example, IP submissions: Australian Red Cross, 8; Australian 
Rehabilitation and Assistive Technology Association, 8-9; CHOICE, 5. 
‘Digital divide’ may be described as the gap between those who are 
digitally included and those who are not. Digital inclusion is based 
on the premise that everyone should be able to make full use of 
digital technologies to manage their health and wellbeing, access 
education and services, organise their finances, and connect with 
friends, family and the world beyond. Australian Digital Inclusion 
Index, Measuring Australia’s Digital Divide: The Australian Digital 
Inclusion Index 2020 (2020). 

IP submission: National Association of Community Legal Centres, 8. 
Standards Australia, What is a Standard? <https://www.standards. 
org.au/standards-development/what-is-standard>. 

Australian Human Rights Commission, Human Rights and Technology 
(Discussion Paper, December 2019) Proposal 23. 

DP submissions: Digital Gap Initiative, 21; Australian 
Communications Consumer Action Network, 19; Standards 
Australia, 7; Confidential submission 42, 18; Commonwealth Bank of 
Australia, 9; Blind Citizens Australia, 4. 

DP submissions: Digital Gap Initiative, 21; Australian 
Communications Consumer Action Network, 19; Blind Citizens 
Australia, 4. 

IP submission: Australian Communications Consumer Action 
Network, 15. 
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See IP submissions: Intellectual Disability Rights Service, 1; 
Australian Rehabilitation and Assistive Technology Association, 12. 
DP Consultation roundtable: 18. 

IP submission: Confidential submission 102, 1. 

IP submission: Intopia, 10. 

IP submission: Ability Research Centre, 2. 

DP submission: Standards Australia, 7. 

DP submissions: Standards Australia, 7; Digital Gap Initiative, 21; 
Blind Citizens Australia, 4. 

DP submissions: Australian Communications Consumer Action 
Network, 19; Digital Gap Initiative, 21. 

Centre for Inclusive Design, PwC Australia, The Benefit of Designing 
for Everyone (May 2019) 1. 

Standards Australia, What is a Standard? <https://www.standards. 
org.au/standards-development/what-is-standard>. 

See Recommendation 24. 

DP submissions: Australian Communications Consumer Action 
Network, 19; Blind Citizens Australia, 6; SAS Institute Australia. 

DP Consultation roundtables: 18, 19, 21; ACMA, Ideal Accessible 
Communications Roadmap (2020) <http://accan.org.au/Ideal%20 
Accessible%20Communications%20Roadmap.pdf> 6. 

ACMA, ‘No Australian Left Offline: ACCAN Pushes for Affordable 
Broadband’ (Media Release, 11 February 2019) 4. 

NBN Co, COVID-19 Education Assistance Package <https://www.nbnco. 
com.au/campaigns/covid-19/education-assistance-package>. 

CRPD art 21. 

Australian Digital Inclusion Index, Measuring Australia’s Digital Divide: 
The Australian Digital Inclusion Index 2020 (2020) 20. 

Australian Digital Inclusion Index, Measuring Australia’s Digital Divide: 
The Australian Digital Inclusion Index 2018 (2018) 10. 

Tony Vinson and Margot Rawsthorne, Dropping off the Edge 2015: 
Persistent Communal Disadvantage in Australia (2015) 48. 

Australian Council of Social Service, Digital Divide Policy Snapshot 
(January 2016) 2. 

IP submission: Australian Red Cross, 8. 

DP submission: SAS Institute Australia, 20. 

DP Consultation roundtables: 17, 18, 19, 20, 21. 

NBN bandwidth per user increased by 31% and there was an 11.9% 
increase in the uptake of the higher speed 100Mbps plans in the 
three months to June 2020: ACMA, ‘Increased Connection to Higher 
Speed Broadband Services During COVID-19’ (Media Release 166/20, 
13 August 2020). 

NBN Co, COVID-79 Education Assistance Package <https://www.nbnco. 
com.au/campaigns/covid-19/education-assistance-package>. 

NBN Co, COVID-19 Education Assistance Package <https://www.nbnco. 
com.au/campaigns/covid-19/education-assistance-package>. 
CHOICE, Bonus Data and Bill Support: What Telcos Are Offering During 
the COVID-19 Crisis (1 April 2020) <https://www.choice.com.au/ 
electronics-and-technology/internet/connecting-to-the-internet/ 
articles/telcos-coronavirus-special-offers-and-support>. 

People with Disability Australia, People with Disability and COVID-19 
(2020) <https://pwd.org.au/wp-content/uploads/2020/06/PWD_and_ 
COVID_report-final.pdf> 2. 

The NDIA states that there are around 4.3 million Australians who 
have a disability and that ‘[w]ithin the next five years the NDIS will 
provide an estimated 500,000 Australians who have permanent 
and significant disability with funding for supports and services’: 
National Disability Insurance Scheme (NDIS), How the NDIS Works 
(14 May 2020) <https://www.ndis.gov.au/understanding/how-ndis- 
works>. 

See National Disability Insurance Scheme Act 2013 (Cth) s 4(11). 
National Disability Insurance Scheme Act 2013 (Cth) s 35(1). 

See NDIS, Assistive Technology Explained (4 December 2020) <https:// 
www.ndis.gov.au/participants/home-equipment-and-supports/ 
assistive-technology-explained#at-assessments>. 

DP submissions: Blind Citizens Australia, 5-6; Australian 
Communications Consumer Action Network, 19; University of 
Technology Sydney, 69. DP Consultation roundtables: 17, 20. 

DP submissions: Blind Citizens Australia, 5-6; Australian 
Communications Consumer Action Network, 19; University of 
Technology Sydney, 69. DP Consultation roundtables: 17, 20. 

IP submissions: National Disability Services, 2; Confidential 
submission 2. 
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DP submission: Blind Citizens Australia, 5. IP submissions: Australian 
Rehabilitation and Assistive Technology Association, 12; Australian 
Communications Consumer Action Network, 8; Confidential 
submission 2, 2. 

DP submissions: Australian Communications Consumer 

Action Network, 19; Blind Citizens Australia, 5. DP Consultation 
roundtables: 17, 20. 

DP submission: Blind Citizens Australia, 5. 

NDIS, Assistive Technology Explained (4 December 2020) <https:// 
www.ndis.gov.au/participants/home-equipment-and-supports/ 
assistive-technology-explained>. 

NDIS, Assistive Technology Explained (4 December 2020) <https:// 
www.ndis.gov.au/participants/home-equipment-and-supports/ 
assistive-technology-explained>. 

NDIS, Low Cost AT for Support Continuity During Coronavirus 
(COVID-19) (4 December 2020) <https://www.ndis.gov.au/ 
participants/assistive-technology-explained/low-cost-support- 
continuity-during-coronavirus-covid-19>. 

NDIS, Low Cost AT for Support Continuity During Coronavirus 
(COVID-19) (4 December 2020) <https://www.ndis.gov.au/ 
participants/assistive-technology-explained/low-cost-support- 
continuity-during-coronavirus-covid-19>. 

P Davidson, P Saunders, B Bradbury and M Wong, Poverty in 
Australia 2018 (Report No 2, ACOSS and UNSW, 2018) 25. 

DP submissions: Canberra Blind Society, 2; Co-Innovation 

Group, School of Information Technology and Electrical 
Engineering, University of Queensland, 2; Public Interest Advocacy 
Centre, 2; Telstra, 4; Consumer Policy Research Centre, 17; 

Digital Gap Initiative, 22-23. DP Consultation roundtables: 17, 18, 
19, 20, 21. IP submissions: S Murphy; Australian Communications 
Consumer Action Network, 9-10; The Centre for Inclusive Design, 1, 
2; Adobe, 3; Google, 6-8. 

IP submission: Google, 6. 

See, for example, IP submissions: Google; Adobe; Digital Gap 
Initiative. 

IP submission: Digital Gap Initiative, 17. 

See, for example, IP submissions: P Harpur, 11; Australian 
Rehabilitation and Assistive Technology Association, 10; Northraine, 
13; National Association of Community Legal Centres, 11; Intopia, 
12; The Centre for Inclusive Design, 5; Adobe, 3; University of 
Melbourne, 12; LexisNexis, 24; University of Technology Sydney, 72. 
WP submission: M Johnson. 

CPRD art 2. 

University of Washington, What is the Difference Between Accessible, 
Usable, and Universal Design? (30 April 2019) <https://www. 
washington.edu/doit/what-difference-between-accessible-usable- 
and-universal-design>; Matt May, ‘The Same, But Different: Breaking 
Down Accessibility, Universality, and Inclusion in Design’, Adobe 
Blog (online, 2 April 2018) <https://theblog.adobe.com/different- 
breaking-accessibility-universality-inclusion-design/>. 

For example, WCAG 2.1 includes accommodations for blindness and 
low vision, deafness and hearing loss, limited movement, speech 
disabilities, photosensitivity, and combinations of these; and some 
accommodation for learning disabilities and cognitive limitations. 
See W3C, Web Content Accessibility Guidelines (WCAG) 2.1 (5 June 
2018) <https://www.w3.org/TR/WCAG21/>. 

OCAD University, Inclusive Design Research Centre, What is Inclusive 
Design <https://idrc.ocadu.ca/about-the-idrc/49-resources/online- 
resources/articles-and-papers/443-whatisinclusivedesign>. 

See especially Australian Government, eSafety Commissioner, Safety 
by Design: Overview (May 2019) <https://www.esafety.gov.au/sites/ 
default/files/201 9-10/SBD%20-%200verview%20May19.pdf>. 

W3C, Accessibility, Usability, and Inclusion (6 May 2016) <http://www. 
w3.org/WAl/intro/usable>. 

Australian Banking Association, Every Customer Counts - Accessibility 
Principles for Banking Services (November 2018). 

IP submissions: P Harpur, 11; S Murphy, 6; WebKeylT, 6; The Centre 
for Inclusive Design, 5. 

See, for example, IP submission: S Murphy, 6; Matt May, ‘The 

Same, But Different: Breaking Down Accessibility, Universality, 

and Inclusion in Design’, Adobe Blog (online, 2 April 2018) <https:// 
theblog.adobe.com/different-breaking-accessibility-universality- 


1011 
1012 


1013 


1014 


1015 


1016 
1017 


1018 
1019 


1020 


1021 


1022 
1023 


1024 
1025 


1026 


1027 


1028 
1029 


1030 
1031 
1032 
1033 


1034 
1035 
1036 


1037 


1038 
1039 
1040 
1041 


inclusion-design/>; W3C, Web Content Accessibility Guidelines (WCAG) 
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